INTERNATIONAL TELECOMMUNICATION UNION CCITT X.293 THE INTERNATIONAL TELEGRAPH AND TELEPHONE CONSULTATIVE COMMITTEE DATA COMMUNICATION NETWORKS OPEN SYSTEMS INTERCONNECTION (OSI) PROTOCOL SPECIFICATIONS, CONFORMANCE TESTING OSI CONFORMANCE TESTING METHODOLOGY AND FRAMEWORK FOR PROTOCOL RECOMMENDATIONS FOR CCITT APPLICATIONS – TEST REALIZATION Recommendation X.293 Geneva, 1992 Printed in Switzerland FOREWORD The CCITT (the International Telegraph and Telephone Consultative Committee) is a permanent organ of the International Telecommunication Union (ITU). CCITT is responsible for studying technical, operating and tariff questions and issuing Recommendations on them with a view to standardizing telecommunications on a worldwide basis. The Plenary Assembly of CCITT which meets every four years, establishes the topics for study and approves Recommendations prepared by its Study Groups. The approval of Recommendations by the members of CCITT between Plenary Assemblies is covered by the procedure laid down in CCITT Resolution No. 2 (Melbourne, 1988). Recommendation X.293 was prepared by Study Group VII and was approved under the Resolution No. 2 procedure on the 17th of January 1992. ___________________ CCITT NOTE In this Recommendation, the expression “Administration” is used for conciseness to indicate both a telecommunication Administration and a recognized private operating agency. γ ITU 1992 All rights reserved. Unless otherwise specified no part of this publication may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and microfilm, without permission in writing from the ITU. PAGE BLANCHE Recommendation X.293 Recommendation X.293 OSI CONFORMANCE TESTING METHODOLOGY AND FRAMEWORK FOR PROTOCOL RECOMMENDATIONS FOR CCITT APPLICATIONS – TEST REALIZATION 1) The CCITT, considering (a) that the Recommendation X.200 defines the Reference Model of Open Systems Interconnection (OSI) for CCITT Applications; (b) that the objective of OSI will not be completely achieved until systems can be tested to determine whether they conform to the relevant OSI protocol Recommendations; (c) that standardized test suites should be developed for each OSI protocol Recommendation as a means to: – obtain wide acceptance and confidence in conformance test results produced by different testers, – provide confidence in the interoperability of equipments which passed the standardized conformance tests; (d) the need for realizing a Means of Testing Implementations Under Test; (e) the need for standardizing the conformance testing process to achieve an acceptable and useful degree of comparability of results of conformance assessments of similar products, unanimously declares the view that the realization of a Means of Testing IUTs should be in accordance with this Recommendation. CONTENTS 0 Introduction 1 Scope 2 References 3 Definitions 4 Abbreviations 5 Test Realization Overview 6 Requirements Concerning Test Realization 7 Compliance Annex A – Additional Guidance on Test Realization 0 Introduction Recommendations X.290 and X.291 define a general methodology for testing the conformance of implementations to OSI protocol specifications and/or transfer syntaxes issued as CCITT Recommendations or International Standards; these Recommendations also put requirements on the production of OSI conformance testing Recommendations and standardized Abstract Test Suites (ATS). Recommendation X.292 defines a standardized test notation, the Tree and Tabular Combined Notation (TTCN), for the specification of a standardized Abstract Test Suite. Once OSI conformance testing standards and standardized Abstract Test Suites are available, the test results obtained by different test laboratories should be comparable, if they base their test operations on the same reference standardized ATS. Recommendation X.294 puts requirements on the conformance assessment process, so that test results can be compared with those of other test laboratories, and can have a wide acceptance. This recommendation in the X.290-Series Recommendations concentrates on the intermediate stage, namely, Test Realization. Before the test preparation can begin, a Means of Testing the Implementation Under Test (IUT) has to be made available. Test Realizers are those organizations which take responsibility for providing such a Means of Testing (MOT). Recommendation X.293 places requirements on Test Realization, to ensure that the execution of test cases reflects the behaviour specified in the reference standardized Abstract Test Suite. In this way, the purpose of the standardized ATS is achieved. This Recommendation is also published as ISO/IEC 9646-4:1991. 1 Scope This Recommendation specifies requirements and gives guidance concerning the realization of a Means of Testing IUTs, in conformance with a reference OSI standardized ATS, specified in compliance with Recommendation X.291. Note – This implies the use of standardized Abstract Test Suites as defined in § 3.6.31 of Recom-mendation X.290. These requirements are limited to those aspects of a Means of Testing which can be mapped on to the abstract testing functions defined in Recommendation X.290, or which are essential to a proper use of the standardized ATS. Such aspects might include a facility to produce conformance logs, or the progression of the PIXIT proforma. Further implementation details of test systems and upper testers are outside the scope of this Recommendation. Acceptance and installation of Means of Testing are outside the scope of this Recommendation. 2 References Rec. X.200 (1988) – Reference Model of Open Systems Interconnection for CCITT Applications (see also ISO 7498). Rec. X.290 (1992) – OSI Conformance Testing Methodology and Framework for Protocol Recommendations for CCITT Applications – General Concepts (see also ISO/IEC 9646-1). Rec. X.291 (1992) – OSI Conformance Testing Methodology and Framework for Protocol Recommendations for CCITT Applications – Abstract Test Suite Specification (see also ISO/IEC 9646-2). Rec. X.292 – OSI Conformance Testing Methodology and Framework for Protocol Recommendations for CCITT Applications – The Tree and Tabular Combined Notation (TTCN) (see also ISO/IEC DIS 9646-3)2). Rec. X.294 (1992) – OSI Conformance Testing Methodology and Framework for Protocol Recommendations for CCITT Applications – Requirements on test laboratories and clients for the Conformance Assessment Process (see also ISO/IEC 9646-5). 3 Definitions For the purposes of this Recommendation, all the definitions given in Recommendation X.290 apply. 4 Abbreviations For the purpose of this Recommendation, the following abbreviations given in Recommendation X.290, § 4, apply: ASP Abstract service primitive ATS Abstract test suite BIT Basic interconnection tests ETS Executable test suite IUT Implementation under test MOT Means of testing (IUTs) OSI Open systems interconnection PATS Parameterized abstract test suite PCO Point of control and observation PDU Protocol data unit PETS Parameterized executable test suite PICS Protocol implementation conformance statement PIXITProtocol implementation extra information for testing SATS Selected abstract test suite SETS Selected executable test suite SUT System under test TTCN Tree and tabular combined notation 5 Test Realization overview 5.1 Test Realization is the process of producing a Means of Testing (MOT) IUTs for conformance to OSI protocol specifications, based on a conformance testing standard and its ATS. 5.2 The MOT is a combination of equipment and procedures that can perform: a)the derivation; b)the selection; c)the parameterization; and d)the execution of the test cases, in conformance with the reference standardized ATS, and can produce a conformance log. 5.3 In the derivation process, the abstract test cases of the reference standardized ATS are converted, so as to be executable on a test system. In the selection process, the appropriate test cases for the IUT are selected, according to the provisions of the PICS and the PIXIT. In the parameterization process, the parameters in the selected test cases are given appropriate values, according to the provisions of the PIXIT (and possibly of the PICS). The MOT is then used in the conformance assessment process of an IUT, resulting in the production of a conformance log. The output of the derivation process is called an “Executable Test Suite” (ETS). It consists of executable test cases. 5.4 Intermediate forms of test suites may or may not be created, depending upon when the derivation process occurs. Such intermediate forms are known as: a)SATS: Selected Abstract Test Suites; b)SETS: Selected Executable Test Suites; c)PATS: Parameterized Abstract Test Suites; d)PETS: Parameterized Executable Test Suites. 5.5 Among these various forms, only the Abstract Test Suites are necessarily tangible. Some MOT may generate the PETS automatically from the reference standardized ATS (given the PICS and PIXIT) at the moment the test cases are actually run. Such a Means of Testing does not exhibit an ETS, nor a SETS, nor a PETS, in a tangible form. Nevertheless, what is executed is always a Parameterized Executable Test Suite. 6 Requirements concerning Test Realization 6.1 Introduction The requirements concerning Test Realization address: a)the MOT as a whole; b)the derivation process, from abstract to executable test cases; c)facilities for producing a conformance log; d)progression of the PIXIT proforma; e)other documentation. 6.2 Requirements concerning the Means of Testing 6.2.1 The Means of Testing an OSI protocol implementation shall be provided in the context of a single standard Abstract Test Suite, in compliance with Recommendation X.291. The Test Realizer shall use only the version of the ATS specification which has the highest standardization status (e.g. Draft Recommendation that is considered stable). The MOT shall provide: a)a realization of the lower tester; b)the specification of the upper tester, insofar as it is required by the abstract test method; c)the realization of the upper tester for the Local test method; d)optionally, the realization of the upper tester for the Coordinated and Distributed test methods; e)the specification of the test coordination procedures in accordance with the requirements specified in the standardized ATS; f)the realization of the test coordination procedures within the test system for the Local test method; g)the realization of the Test Management Protocol within the lower tester for the Coordinated test method. (See § 7.4 of Recommendation X.290 and § 12.3 of Recommendation X.291.) 6.2.2 The MOT shall include either the executable test cases derived from the test cases of the reference standardized ATS, or a means of deriving them. The MOT shall be realized in compliance with the semantics of the test notation chosen in the reference standardized ATS. The MOT shall provide a means of selecting and parameterizing the test cases (whether they are at the abstract level or at the executable level), according to the appropriate PICS and PIXIT information provided with an IUT (see §§ 7.3 and 7.4 of Recommendation X.294). 6.2.3 The MOT shall provide a facility for selecting the capability or behaviour test cases mentioned in the list of Basic Interconnection Tests (BIT list), if such a list is specified in the reference standardized ATS, and shall provide a facility for running them initially altogether, before the capability and behaviour tests. The MOT shall also provide a facility for omitting those test cases indicated in the BIT list from the set of test cases selected for the capability and behaviour tests. 6.2.4 The MOT shall include the capability of executing the parameterized executable test cases which result from the derivation, selection and parameterization processes. 6.2.5 The Test Realizer shall provide a statement of conformance of the MOT to the reference standardized ATS, indicating any subset of the ATS that is not supported (see § 6.3.4). The Test Realizer shall identify all restrictions for test execution required by the MOT beyond those stated in the reference standardized ATS (e.g. limiting value ranges provided in the PIXIT). Note – The Test Realizer should note the requirements for a comprehensive testing service stated in the standardized ATS. The Test Realizer may wish to develop an MOT for each of the required abstract test methods in order that a test laboratory may provide a comprehensive testing service. 6.2.6 The MOT shall provide a facility for generating a conformance log (see § 6.4). 6.3 Requirements concerning ETS derivation 6.3.1Introduction Requirements in § 6.3 shall apply to all Executable Test Suites, including SETS or PETS, whether tangible or not. 6.3.2Conformance to the reference standardized ATS An ETS shall be derived from a single reference standardized ATS. For an ETS to conform to the reference standardized ATS, it shall comply with the requirements stated in § 6.3.3 to 6.3.5 below. It shall also conform to the requirements stated in the reference standardized ATS itself, and in the other X.290-Series Recommendations, if applicable (e.g. TMP). 6.3.3Correspondence between ATS and ETS Each executable test case shall be the realization of a single Abstract Test Case, and shall be selectable for execution on an individual basis. All sequences of test events comprising an abstract test case shall be capable of being realized in the executable test case. The test purpose and verdict assignments of each Abstract Test Case shall be maintained in the corresponding executable test case. The MOT shall not perform checking on the validity of PDU parameters received from the IUT in addition to that which is defined in the Abstract Test Case. Any further checking which the test system might be capable of performing is outside the scope of this Recommendation and shall not contribute to the verdict assignment for each test case. Test group relationships defined in the reference standardized ATS shall be maintained in the ETS. Each test group composed of a named set of test cases in the reference ATS shall be represented in the ETS as a named set of executable test cases. The standardized ATS includes a mapping of the abstract test case(s) to the PICS and partial PIXIT proforma entries (see § 15 of Recommendation X.291). This mapping shall be maintained in the ETS. 6.3.4Subsetting the ATS The ETS derivation process generally results in the derivation of all abstract test cases of the reference standardized ATS. However, it may be acceptable to derive an ETS for certain subsets of the ATS. If a subset is created, the exclusion of a set of test cases shall be consistent with the test selection process for an IUT, with respect to the mapping between the PICS (and PIXIT) proforma entries and the test cases in the ATS. Note – This means that the test cases which are mandatory for all IUTs would always be included in the subset, but the Test Realizer can choose not to realize particular sets of test cases which are optional or conditional and therefore, will not be required to test particular classes of IUTs. Thus the subset of the reference standardized ATS which is realized shall be equivalent to one or more of the potential SATS. 6.3.5Derivation process independence In the MOT, the derivation process shall lead to the same PETS being executed for a given IUT, regardless of when the derivation process occurs relative to the selection and parameterization processes. Note – See Figure A-1/X.293. The application of the selection and parameterization processes for a particular IUT is the responsibility of the test laboratory, in the Test Preparation phase. 6.4 Requirements concerning conformance logs As stated in § 6.2, the MOT shall provide a facility for generating a conformance log. A conformance log is a human-readable record of information produced as a result of a test campaign, sufficient to record the observed test outcomes and verify the assignments of test verdicts. This information combines the observations of the actual test events which occur when the PETS is run against an IUT, with information which relates those events to the abstract test cases concerned. A conformance log may be used in the production of conformance test reports and in the resolution of disputes and queries which may arise during or as a result of the conformance assessment process. A conformance log shall include: a)a unique identification of the conformance log that includes time and date of the start of the execution of the PETS; b)an identification of the MOT, date of origin, version number, and ETS identification (if any); c)an indication of the start and end of run of each test case, including a unique reference to the abstract test case as specified in the ATS (e.g. TTCN test case reference or test identifier); d)the PDUs sent by the lower tester to the IUT, and received by the lower tester from the IUT, including a record of the detailed information contained in the PDU parameters and user data; e)the abstract test events, as specified in the relevant abstract test case; these include all abstract service primitives observed by the lower tester, all test events received via the test coordination procedures by the lower tester containing information from the upper tester, and an identification of the relevant Points of Control and Observation (PCO); f)an indication of the result for each test case; this will be verdict assignment, abstract or executable test case error, or abnormal test case termination; g)a time stamp or ordering sequence for all test events logged by the lower tester in the order that they are observed; h)any additional information required by the reference standardized ATS. Note – An example of h) above is when an abstract test case written in TTCN, specifies that preliminary result information (in the verdict column) or labels (in the label column), shall be recorded in the conformance log if the corresponding test event occurs. A conformance log shall display all names, abbreviations and values, using the terminology and conventions defined in the protocol specification, transfer syntax (if any), or reference standardized ATS (with precedence given to the first two named). The MOT shall have the ability to produce the conformance log on paper. It is also recommended that a machine-readable form of the conformance log be made available, with equivalent contents. Note – See Annex A, § A.3, for guidance on the production of conformance logs. 6.5 Requirements on the progression of the PIXIT proforma The partial PIXIT proforma, as specified in the reference standardized ATS, shall be progressed to take into account the Means of Testing. To achieve this, the Test Realizer shall augment the partial PIXIT proforma by adding those additional questions that need to be answered in order to prepare the MOT for a particular IUT. The Test Realizer shall include in the augmented partial PIXIT proforma all the information concerning the realization of the reference standardized ATS, which the client needs for completing the PIXIT. The Test Realizer should refer to Recommendation X.294, Annex C and produce the augmented partial PIXIT proforma in compliance with this annex. The resulting augmented partial PIXIT proforma shall be provided to the test laboratory, in order that it can fulfill its requirements as specified in § 6.4.3 and Annex C of Recommendation X.294. 6.6 Requirements concerning other documentation Documentation shall accompany the MOT, in order to enable the test laboratory to perform test operations in conformance with the reference standardized ATS, and in compliance with Recommendation X.294, with respect to information to be provided to the client. The documentation shall include: a)identification of the MOT, date of origin, version number, and ETS identification (if any); b)name and version number of the CCITT Recommendation or International Standard for the protocol specification (and the service definition if appropriate); name and version number of the reference standardized ATS, together with lists of technical corrigenda which have been taken into account; c)description of the MOT (see § A.4 for guidance); d)specifications of the test coordination procedures and of the upper tester, as and when required by the reference standardized ATS; e)the test cases, if any, which cannot be executed due to limitations in the MOT; Note – Such limitations should be exceptions, and should occur only if particular abstract test cases could not feasibly be realized. f)description of those procedures for test execution which are to be performed by the test laboratory and/or the client, and which are specific to the MOT; g)statement of conformance to the reference standardized ATS; h)statement of compliance with this Recommendation; i)guidance for interpreting the conformance logs. If Test Realizers detect errors in any of the abstract test cases or detect any abstract test case which addresses erroneous or ambiguous requirements in the relevant OSI protocol specification, then the Test Realizers shall identify those test cases in the documents accompanying the MOT. Note – The Test Realizers should also forward defect reports which identify the problem(s) to the proper CCITT or ISO/IEC committee. 7 Compliance A Means of Testing IUTs complies with this Recommendation if, and only if, all the requirements stated in § 6 are satisfied. Note – The primary means of verifying that a MOT implements the four functions associated with Test Realization (i.e. derivation, selection, parameterization, execution) resides in the conformance log. ANNEX A (This annex does not form an integral part of this Recommendation) Additional guidance on Test Realization A.1 Additional guidance on the MOT A.1.1Introduction This annex gives guidance on how the three abstract testing functions defined in § 7.4 of Recommendation X.290, namely the lower tester, the upper tester and the test coordination procedures, can be defined or realized in an MOT. Note – A test system should be able to accommodate several MOTs. A.1.2The realization of the lower tester A.1.2.1 For every abstract test method defined in Recommendation X.291, the primary focus for coordination and control of the test is the lower tester. The functions of the lower tester are to: a)run executable test cases which are derived from abstract test cases; b)produce verdict indications in accordance with the reference standardized ATS; c)control and observe the test events which are included in an abstract test case (these events include generation and receipt of PDUs, abstract service primitives, generation and receipt of Test Management PDUs, events related to test coordination procedures). The lower tester is part of an independent real system, referred to as the test system. Both the test system and the SUT provide the underlying service, below the lowest layer of protocols in the IUT. A.1.2.2 The OSI entities in the lower tester, peer of the IUT, can be designed according to different techniques, for example: a)Encoder/decoder – Simply encodes and decodes the ASPs and PDUs as required for the test case being run, without being an implementation of the protocol in question. b)Enhanced implementation – An implementation of the protocol concerned, modified by the addition of an error generator, configuration module or similar device to ensure that invalid or unusual ASPs or PDUs can be generated as required by the test case being run. A.1.3The realization of the upper tester An MOT provides a realization of, or a specification of, the functions of an upper tester, according to the abstract test method used in the reference standardized ATS. The upper tester can take different forms, for example: a)a software implementation of the upper tester (which may or may not be independent of the design of the SUT and IUT), installed in the SUT above the IUT, with a mapping region that interfaces with the local realization of the ASPs; b)a human operator: the functions of an upper tester are performed by a person having access to a user interface that maps onto the IUT service boundary and accesses and manipulates the realization of the appropriate ASPs; c)a notional upper tester, i.e. the upper layers of the SUT are used to realize the functions of the upper tester, without any additional mechanism being installed (this can be used to realize the Remote abstract test method only). A.1.4 The realization of the test coordination procedures There are many ways in which the lower tester can interact with the upper tester, e.g. with or without synchronization, with or without using a communication channel additional to the one used between the lower tester and the IUT, etc. Several common types of implementation can be identified: a)Human operator – The functions of an upper tester are performed by a person having access to a user interface that maps onto the IUT service boundary; this operator synchronizes with the lower tester, the progress of which can be detected by various means; e.g. via a set of prompting messages from a user interface of the lower tester. b)Scenario interpreter – The upper tester is realized by a remote scenario interpreter; it takes its instructions from files generated in conjunction with the lower tester installation, with a mapping region between it and the IUT service boundary. c)Test Management Protocol – The upper tester is synchronized with the lower tester by means of a test management protocol, which uses the service provided by the IUT and its underlying layers, and the corresponding functions of the lower tester. A.2 Additional guidance on the ETS derivation process A.2.1 Overview The derivation process can occur: a)during test realization; b)during the installation of the MOT by the test laboratory; c)during test preparation, intermixed with the selection and the parameterization processes, for a particular IUT; d)during test operations, as a result of interpreting or compiling the reference ATS. Figure A-1/X.293 illustrates the many possibilities of combining the test derivation, selection, parameterization and execution processes which are described in § 5, and for which requirements are specified in § 6. Fig. A-1/x.293= 9.5 cm A.2.2 Inputs to ETS derivation The Test Realizer has to consider the following inputs: a)the reference standardized ATS for a particular OSI protocol, based on a particular abstract test method, and containing the specifications of the test coordination procedures; b)the PICS proforma for the OSI protocol; c)the partial PIXIT proforma, normally attached to the reference standardized ATS. A.2.3 ETS maintenance Once the capability of executing a PETS has been implemented in an MOT, and the MOT is in use, the test realizer may receive problem reports from the test laboratories. Problems may arise with the execution procedures, or with conformance to the reference standardized ATS. The Test Realizer should in such circumstances make available the appropriate corrections. The Test Realizer should also provide an update of the MOT every time there is an update to the reference standardized ATS. A.3 Additional guidance on conformance logs In order to produce a conformance log it is necessary to: a)record the actual test events in their order of occurrence during the execution of the PETS; b)analyse this information with respect to the relevant selected and parameterized abstract test cases, mapping the actual test events onto the abstract test events and recording all other necessary information. There are requirements only on the information to be recorded in the conformance log, and how it is to be expressed. The analysis of the ordered list of actual test events can be built into, and performed after the execution of each executable test case; it may also be performed as a distinct process after the execution of the PETS, or performed by some combination of these techniques. The means of performing this analysis, and the timing of this analysis with respect to the execution of the PETS, are not standardized. As specified in § 6.4, the MOT shall have the ability to produce the conformance log on paper. It is also recommended that a machine-readable conformance log, with equivalent contents, be made available. The process of producing the conformance log can be illustrated conceptually as in Figure A-2/X.293. Note – It is intended that the test laboratory retains, as a minimum, either the ordered list of actual test events or the machine readable version of the conformance log. Fig. A-2/X.293 = 9,35 cm A.4 Additional guidance on documentation A.4.1 Introduction In addition to the requirements specified in § 6.4 and 6.5, the preparation of the following documents is recommended: a)test system information; b)MOT description; c)test laboratory client information; d)test laboratory operating instructions. A.4.2Test System Information Document An MOT is adapted to a specific test system. This document should contain the following information related to that test system: a)equipment (test system); b)operating system name and version number (test system); c)name and version number of lower tester; d)name and version number of upper tester, if any; e)equipment and/or procedures necessary to link the lower tester to the IUT for testing purposes (i.e. -service); f)equipment and/or procedures necessary to link the upper tester, if any, to the IUT for testing purposes; g)name, location and contact information for the organization responsible for maintaining and giving advice on the MOT and the ETS. A.4.3Means of Testing Description document This document should contain descriptions of the following aspects of the MOT, with respect to the reference standardized ATS : a)Lower tester – A description of the executable test notation and its mapping onto the abstract test notation (for example, onto TTCN). A description of how the ASPs are controlled, observed and stored, and a demonstration that the method chosen realizes the sequencing rules laid down in the abstract test cases. b)Upper tester – A description of how the ASPs are controlled, observed and stored – except for the Remote test method – to show how the requirements on upper tester functions are met. c)Test coordination – A description of the mapping of the test coordination procedures to their realization; the requirements for this are specified in the reference standardized ATS. d)Selection process – A description of the use of the PICS and PIXIT in selecting abstract test cases suitable for testing the IUT. e)Parameterization process – A description of the use of the PICS and PIXIT in parameterizing executable test cases for testing the IUT. f)Facilities to produce a conformance log. A.4.4Test Laboratory Client Information document In this document, the Test Realizer should provide the following information to enable the test laboratory to inform its client how to prepare the SUT for testing: a)Upper tester – If this component is supplied, a description of how to map its interface to the appropriate realization of the service boundary, any assumptions made about implementation of the service definition, or any assumptions made about the capabilities or resources available within the SUT; if the upper tester is not supplied, a description of how to implement it should be included; such a description includes the Test Management Protocol (TMP) if there is one. b)Test coordination – What the client has to do to implement the test coordination procedures – a description of how to do any manual coordination between the SUT and the lower tester if this is necessary – any relevant timing information, e.g. the expected performance of the Test Management Protocol; c)Underlying service – Indicate that the client has to provide a sufficiently reliable (N – 1)-service and, as far as possible, explain how this is to be achieved (without referring to a particular computer). A.4.5 Test Laboratory Operating Instructions document In this document, the Test Realizer should provide information that will assist and guide the test laboratory in the execution of tests on the MOT, the diagnosis of problems and the re-running of tests if necessary. This should include: a)Test preparation – How to use PICS and PIXIT to perform test selection and parameterization on the MOT. b)Test execution – A description of how to run tests on the lower tester, and how to analyse the results. c)Execution control – The definition of the level of detail of control over the execution of test cases; the operating instructions should describe how test cases are executed and thereby implicitly define how many test cases may be executed as a single execution unit; an extreme case is when there is one single command for the entire test campaign (Basic Interconnection Tests, capability tests, and behaviour tests); the other extreme is when there is a command for every single test step in every single test case in the ETS. d)Conformance logging – Control of its execution; how the contents of the conformance log can be mapped back to the standardized test events in the reference ATS specification. e)Upper tester – A description of any initial confidence tests to be performed on the upper tester and how to obtain the stored test events from the upper tester. f)Test coordination procedures – A description of how to do any manual coordination between the lower and the upper testers if this is necessary. _______________________________ 1) Recommendation X.293 and ISO/IEC 9646-4, Information technology – Open Systems Interconnection – Conformance Testing Methodology and Framework – Part 4: Test Realization, are technically aligned. 2) To be published.