Working Implementation Agreements for Open Systems Interconnection Protocols: Part 24 - Conformance Testing Output from the December 1993 Open Systems Environment Implementors' Workshop (OIW) SIG Chair: Eva Kuiper, Hewlett Packard Workshop Editor: Brenda Gray, NIST PART 24 - Conformance Testing December 1993 (Working) Foreword This part of the Working Implementation Agreements was prepared by the Conformance Testing Special Interest Group (CTSIG) of the Open Systems Environment Implementors' Workshop (OIW). See Part 1 - Workshop Policies and Procedures of the "Draft Working Implementation Agreements Document." Text in this part has been approved by the Plenary of the above- mentioned Workshop. Future changes and additions to this version of these Implementor Agreements will be published as a new part. Deleted and replaced text will be shown as struck. New and replacement text will be shown as shaded. ii PART 24 - Conformance Testing December 1993 (Working) Table of Contents Part 24 - Conformance Testing . . . . . . . . . . . . . . . . 1 0 Introduction . . . . . . . . . . . . . . . . . . . . . . 1 1 Scope . . . . . . . . . . . . . . . . . . . . . . . . . 1 2 Normative References . . . . . . . . . . . . . . . . . . 1 3 Status . . . . . . . . . . . . . . . . . . . . . . . . . 1 4 Errata . . . . . . . . . . . . . . . . . . . . . . . . . 1 5 Guidelines on Interpretation of Disputed Test Cases . . . 1 5.1 Abstract test cases . . . . . . . . . . . . . . . . 1 5.2 Executable test cases . . . . . . . . . . . . . . . 2 5.3 Static analysis and test case selection . . . . . . 2 6 Guidelines on the Choice of PICS . . . . . . . . . . . . 2 7 CT SIG Resolution for FTAM . . . . . . . . . . . . . . . 2 8 Guidelines for PCTR Test Campaign Summary . . . . . . . 3 9 Resolutions which apply to formal test campaigns . . . . 3 9.1 Testing of collocated MHS '88 elements . . . . . . . 3 9.2 Testing of MHS '88 systems for 1984 conformance . . 3 iii Part 24 - Conformance Testing 0 Introduction (Refer to Stable Implementation Agreements Document) 1 Scope (Refer to Stable Implementation Agreements Document) 2 Normative References (Refer to Stable Implementation Agreements Document) 3 Status This material is current as of December 6, 1993. 4 Errata Errata will be reflected in replacement pages of Version 7, Stable Document. 5 Guidelines on Interpretation of Disputed Test Cases 5.1 Abstract test cases The guidelines are given as follows: a) The Certification/Registration body shall present to the Conformance SIG the list of disputed test cases prior to the workshop; b) If the Conformance SIG is unable to resolve specific interpretations, the problems shall be discussed with the relevant protocol SIGs for resolution at the same workshop; c) If the OIW is unable to resolve an issue, then the OIW will refer the problem to the relevant standards body. In such a case the OIW will recommend to all relevant Certification bodies that the test case be considered as deselected until it is resolved; d) In the case where a resolution is reached by the OIW, the new interpretation shall be distributed to the 1 PART 24 - Conformance Testing December 1993 (Working) Certification bodies, relevant standards body, MOT suppliers, and the test case maintenance authority. 5.2 Executable test cases These problems may be brought before the OIW at the discretion of a product supplier, test system supplier, test lab, or Certification Authority. Resolutions will be determined in the same way as for Abstract Test Case problems. 5.3 Static analysis and test case selection Disputes regarding static analysis and test case selection will be handled as above in the case of Abstract Test Cases. 6 Guidelines on the Choice of PICS SIGs are responsible for referencing the appropriate base standard PICS proforma for the protocols used by their specific profiles. The SIGs are also responsible for producing the International Profile Requirements List(s) for their specific parts in the Implementors Agreements. Where an internationally harmonized PICS proforma exists, it shall be used. In the absence of an available PICS proforma, the SIGs are encouraged to use the guidelines stated in ISO 9646 to define a PICS proforma and arrange to have it submitted to the appropriate standards body. The consequence of not providing for an internationally harmonized PICS proforma are that implementors may have to deal with multiple PICS proformas for the same protocol. 7 CT SIG Resolution for FTAM The PICS reflects the product. The product being tested is the protocol machine and the necessary software to fulfill the functionality indicated in the PICs. 2 PART 24 - Conformance Testing December 1993 (Working) 8 Guidelines for PCTR Test Campaign Summary Refer to the Stable Agreements Document. 9 Resolutions which apply to formal test campaigns 9.1 Testing of collocated MHS '88 elements Part 8 of the SIA states that the UA, MS, and MTA configuration is not restricted; any of these components may be collocated, although they are depicted as logically separate. In the case of a collocated UA and MS, a proprietary interface may be used instead of P7. In the case of a collocated MS and MTA, or a collocated UA and MTA, a proprietary interface may be used instead of P3. In the absence of P3 and P7, These components (MS, UA, and MTA) must be tested and registered together. 9.2 Testing of MHS '88 systems for 1984 conformance For TCs Rts 1.2.2.3, 51.2.2.3, 55.2.2.1, 56.2.2.1, 56.2.2.2, 56.2.2.3, and 56.2.2.4, since normal mode should be the default mode, do the following: a) If implementation is statically configurable, configure as MHS 84; b) If implementation is only dynamically configurable: 1) The test cases shall be run, and where possible, a manual verdict assessment performed in consultation with JITC; 2) If manual verdict is not possible, a manual verdict assessment should be performed in consultation with JITC. 3