ABSTRACT. The EUMETSAT ground segment for the Meteosat Transition Programme (MTP) entered into operation in December 1995. The paper presents the mission data processing aspects of this ground segment. It addresses the acquisition of the image data in three spectral channels, its geometrical correction, quality control, dissemination, and the extraction of meteorological products from the image data. Firstly, the underlying concepts, like the real-time processing and the data-driven implementation are presented. The major mission data flows are defined and the data circulation through the ground segment and to the end users is described, including summary timeliness figures. The distributed mission data processing architecture is presented, including the application-specific redundancy schemes that were retained to comply with the high availability requirements. The flexibility and the potential for upgrade of this architecture is demonstrated by the real case of the a-posteriori addition of the Meteosat-5 and Meteosat-6 anomaly correction in the ground segment.
The development of the MTP (Meteosat Transition Programme) ground segment started in the course of 1992 and was concluded by the operational validation that took place during 1995. The operations of the Meteosat satellite were taken over officially on 1st December 1995. The present paper summarises the initial concept for the mission data processing in this ground segment and the retained technical solutions. It further presents the experience gained from such a development and provides some initial results related to the mission data processing part of the MTP ground segment for the first months of operation. In the following, the term mission data is used to represent all data related to the Meteosat image data, the term processing of mission data including the direct derivation of meteorological products from these images.
The MTP ground segment was designed to take over the control and the processing of the mission data for the current generation of Meteosat satellites. These spin-stabilised satellites nominally located at 0° longitude on the geostationnary orbit, perform acquisition of the full Earth hemisphere in the three spectral bands: Visible (broadband, 2.25 km resolution), Infrared (6.4 µm, 4.5 km resolution) and Water Vapour (11.5 µm, 4.5 km resolution). The MTP ground segment encompasses all the functionality required to perform the mission control of the Meteosat satellites, including satellite and ground segment M&C, Mission Analysis, Mission Planning and Flight Dynamic. It further provides all facilities to process, quality control, distribute, archive and retrieve the image data and to extract meteorological products from it. Figure 1. presents the decomposition of the ground segment in separately-procured Facilities, each of them dedicated to certain basic functions:
Primary Ground Station (PGS): (representing also the Backup Ground Station), is providing all functionality for data reception from and sending to the Meteosat S/C. Core Facility (CF): provides all Mission Control functionalities including the following Mission Data processing functions: reception and acceptance of the raw image data, image rectification including correction for the S/C dynamical behaviour, orbit and attitude offset and instrument effects, image quality assessment, verifying the radiometric and geometric image quality, dissemination of image information and interface to GTS for the distribution of meteorological products, and finally processing of Data Collection Platform (DCP) data. Meteorological Products Extraction Facility (MPEF): generates the operational range of meteorological products from the rectified Image data (Cloud Motion Winds, Sea Surface Temperature, Upper Tropospheric Humidity, Cloud Analysis, Cloud Top Heights, Precipitation Index, ISCCP Data Set , Climate Data Set). Meteorological Archive and Retrieval Facility (MARF): provides the central archive for the Meteosat image data and the generated meteorological products, and the retrieval service to the users. User Station Display Facility (USDF): Monitors the quality of the reception of the information disseminated via the Meteosat dissemination channel.
A fundamental requirement from the users of Meteosat mission data is, beside the high availability of the provided service, a minimised delay between the acquisition of the raw image and the generation/distribution of the information (images and products) for further processing. The basic concept of the mission data processing in the MTP ground segment is an attempt to comply to both aspects. From the point of view of the minimised processing delay, an ideal precondition for real-time processing is provided by the fact that the raw image is acquired by the S/C in a line by line manner. The processing of the mission data in the different facilities can be defined in such a manner that the algorithms are applied progressively to the data as it gets available, in a data-driven manner. The advantage of this type of processing is obviously that there is no longer the need to wait for the completion of a full image (nominally 30 minutes) before starting to process the data. The actual delay of the processing is in fact reduced to the contribution of the minimum mathematically required input data size plus some margin for internal data access and transfer.
Another important side-effect of this data-driven processing is the resulting smoothing of the system load (usage of CPU but also of data transfer and data accesses) as a function of time, avoiding sudden peaks of activity and data transfers. As a consequence, the system has shown to be more robust to all kinds of interactions, for example with the operator via man-machine interfaces, or to external perturbations requiring a temporary increase of the system load. The subsequent sections will present the initial operational experience gained from developing and operating such a system.
As a direct consequence of the data-driven and progressive processing, the circulation of the mission data in the MTP ground segment is based on a number of streams of mission data transferred between the processing facilities (DCP and MDD data are omitted here for simplification).
Raw Image The raw image is acquired by the S/C line by line from South to
North in cycles of 30 minutes, with an average downlink data
rate of approximately 200 Kbps. The data is received by the PGS
and transferred to the Control Centre where the data is used to
derive the rectified image data and is finally transferred to
MARF in order to be archived. Raw image files and photographic
prints can be retrieved from MARF by the users.
Rectified The Rectified image provides a one-to-one relation between the
Image pixel position in the image grid and the geographical
co-ordinates of the measured radiance. It is generated by the
Image Processing Element of the Core Facility in near-real-time
with minimised delay, still respecting very good relative and
absolute accuracy. These images are quality-assessed, and
disseminated via the dissemination function, together with
radiometric calibration information. In parallel, the data are
provided to the MPEF where extraction of meteorological
products is performed.
MPEF There are different types of MPEF products, some being clearly
Products less time critical than the others. In the current paper we
will focus on the most complex and time critical products, like
the Cloud Motion Winds or the Sea-Surface Temperatures. The
products are generally extracted using the information from all
three channels, on the basis of individual image segments
(currently 32x32 pixels) and with full disk coverage within the
55° geocentric angle around the sub-satellite point.
Figure 2. presents a timing diagram for the image data flows. It defines the timing for the acquisition of the raw image, the generation of the rectified image, its transfer to the Meteorological Products Extraction Facility and to the dissemination function. The timing for the potential early dissemination of the A format is presented. Finally, the timing for the transfer of the raw image file corresponding to an entire Earth Image to the archiving function in MARF is shown. One can notice that the limitations of the system are currently mainly concentrated at the interfaces with the users community, where a major aim was to initially provide continuity of the existing service and a very smooth transition when the MTP ground segment entered into operations.
To support the data-driven, progressive processing approach, the processing architecture and the allocation of functions within the ground segment had to be carefully analysed, especially for functions that are new or modified with respect to the previous operational system (MOP Programme).
In the example of the generation of the most complex MPEF product, the Cloud Motion Winds, the basic approach was to perform all computation-intensive tasks, mainly the masked correlation with the previous image, in a progressive manner, therefore reducing the peak load on the MPEF system. The two wind components extracted from two subsequent pairs of images are then combined to form the CMW product, based on a triplet of images. Figure 3. schematically presents the CMW extraction concept for the baseline extraction configuration and for a maximum extraction rate, using triplets of subsequent images.
For all aspects related to correction of undesirable effects that are likely to occur in operational systems, these were corrected as close as possible to the source of the data, most of them already in the image data acceptance. Further, in order to allow the system to operate even in degraded mode and to be protected against the occurrence of missing or corrupted data, a feature has been built in, that attaches validity information to each single data unit (image lines or products from a segment) flowing through the system. When information is derived from a given data unit, this validity information is also processed, and allows to closely control the impact of a possible corruption of the input data on the derived data like meteorological products, therefore allowing to support the quality control of the extracted information before providing it to the user community.
The data-driven processing presented above is orientated towards providing minimum delay and most regular load on the system. To achieve the very high availability figures that are required from this operational system, a distributed hardware and communications architecture based on workstations has been selected for the facilities of the ground segment. The example of the most complex of these facilities, the Core Facility is presented here.
The MTP Core Facility is based on a distributed processing architecture using VAX workstations communicating via an Ethernet LAN and, for the high rate data flows, through two interconnected, internally redundant FDDI rings. This architecture has been selected for the incremental growth possibility it offers; the use of commercial off-the-shelf equipment and of different levels of redundancy allows for a high availability and for reduced maintenance delays and costs. A diagram of the Core Facility architecture is presented in Figure 4. One operational processing chain and one validation chain are foreseen. The operational chain allows the control of up to three satellites in parallel and performs the complete imaging mission data processing and dissemination for up to two satellites in parallel. Due to the high availability requirements that apply, the operational chain is a fully redundant one, every workstation working either in warm or cold redundancy mode, depending on the criticality of the application, with one backup workstation. The operational chain consists of 16 workstations of the type VAX 4000-90 (32.8 SPECmarks CPU) and 14 workstations of the type VAX 4000- 60 (12 SPECmarks CPU), this including the backup (redundant) machines. The validation chain is a reduced version of the operational chain and can be used e.g. for training, simulations and verification of new algorithms in parallel to the operational processing, this being made possible by duplication of the data flow from a given satellite. The validation chain consists of 8 VAX 4000-90 and 7 VAX 4000-60 workstations. This validation chain is currently being upgraded for extended mission support.
The MTP ground segment has to provide the operational service for many years, including the commissioning of potentially up to two MTP satellites. During this period, a number of modifications and extensions will also have to be incorporated in the system to respond to the demands of the users of the Meteosat mission data.
The software and hardware architecture based on Software Package Units, distributed on a set of workstations, has already proven its potential for flexibility for the real case of the correction of the Meteosat-5 and Meteosat-6 anomalies in parallel to the operational validation activities of the baseline ground segment. The images from Meteosat-5 are affected geometrically by a rotating lens in the IR and WV channels, requiring specific real-time correction. Meteosat-6 suffers from grey-level variations in the same channels that require complex radiometric processing to be removed. In both cases, the algorithms for the correction of the effects were defined very late (for Meteosat-6, the final algorithm is not yet fully operational), at a point in time when the MTP ground segment design and facility integration phases were already completed, and had therefore to be added to the existing system. The figure 5 presents the implementation of the corrections that were applied to the Image Processing Element within the Core Facility. The Meteosat-5 correction is a classical case of complex S/W modification, but with only minor interfaces upgrade (interface to MARF). The Meteosat-6 correction required the addition of a specific workstation and of a new Software Package Unit with added data flows over the communications infrastructure. The impact is currently a slightly higher delay in the processing and a modification to the MARF interface which was in fact implemented together with the modifications required by the Meteosat-5 correction.
The experience gained from this architectural upgrade of the MTP ground segment has shown that is was less cumbersome than initially anticipated to integrate workstations with specific extensions of functionality or new functionalities into the system. In particular, the data-driven processing performed subsequently on the data flow adapted naturally to the slightly modified timeliness of the data transfer. The most critical dataflow, being the transfer of the rectified image in a progressive manner to MPEF, in fact presented only very minor integration problems.
The concepts and the architecture presented above have been implemented, tested, validated and are in operational use since December 1995. The figures 6 to 9 present a summary of the operational performance and availability before and after the taking over of the operations. The major functions presented are the image acquisition and rectification, the dissemination function and the extraction of meteorological products. The summary mission performance (Figure 6) covers the monthly average performance for the first quarter 1996. Figures 7 and 8 show long term monthly performances for the past year(s) of operations. On these diagrams, the smoothness of the operational transition between the MOP ground segment and the MTP Ground Segment in December 1995 can be seen. For the meteorological products extraction, Figure 9 presents the statistics for all products for a period of one month (January 1996).
Figure 6: Summary Performance (1/96 - Figure 7: Image Processing Performance
3/96)
(4/95 - 4/96)
Figure 8: Met. Products Extraction Figure 9: Dissemination Performance
Performance (4/94 - 4/96)
(1/96)
7. ACKNOWLEDGEMENTS
The authors wish to express their gratitude to all the persons who contributed to the present paper. Furthermore, all members of the team having contributed through their work, ideas and experience to define and realise the mission data processing part of the MTP ground segment as it is, deserve many thanks.