August 9, 1989 TO: X3T9.3 Fiber Optic Study Group Members FROM: Roger Cummings SUBJECT: FIBER CHANNEL WORKING GROUP MINUTES Please find attached a draft of the minutes of the ANSI X3T9.3 Fiber Channel Working Group of July 17 and 18, 1989. Note that there are also fourteen Attachments to the minutes that relate to presentations at the meeting. Note that this package of minutes and attachments will be directly mailed only to those persons who have attended at least one of the two most recent working group meetings. The full package of minutes and attachments will continue to be included in the regular X3T9.3 mailing that results from the plenary meeting following the working group (in this case the October mailing). Thus interested parties that are unable to attend the working group meetings are strongly advised to subscribe to that mailing. The next working group meeting will be held on the Wednesday of the August plenary week (August 23) at the Clarion Hotel in Colorado Springs, CO. See the hotel meeting board for the name of the meeting room. It is expected that the meeting will start at 830am and end at 300pm to allow those travelling eastwards to head home. A schedule of X3T9.3 meetings (both plenaries and working groups) is attached, along with a meeting notice for the October plenary to be held in Raleigh, NC. If there are any corrections required to, or omissions noted from, the minutes I can be reached as follows: Phone: (303) 673-6357 (Business) (303) 665-0761 (Home) Telex/MCI Mail: (650) 289-5060 Fax: (303) 673-5891 Regards Roger Cummings Senior Engineer Subsystems Controller Development #nat0/rc MINUTES OF THE EIGHTH FIBER OPTIC WORKING GROUP MEETING The Eighth meeting of the ANSI X3T9.3 Fiber Optic Working Group was hosted by Jim Smith of Tandem Computers at their facility on Ridgeview Court in Cupertino, CA on July 17 and 18, 1989. A total of 42 people attended, as follows: 3M FIBER OPTIC PRODUCTS H. Robert Carter AMD John Jaeger Jim Kubinec Paul Scott Ron Treadway Yun-Che Wang AMDAHL Rich Taborek Kam T. Tam AMDAHL/KEY Ed Cardinal AMP Charles Brill AT&T MICROELECTRONICS Phillip Fraley AVANTEK Gary LaBelle CORNING INC. Kevin Able CRAY RESEARCH INC. John Renwick Wayne Roiger DIGITAL EQUIPMENT CORP. Chris Baldwin Don Knudson ENDL I Dal Allan FUJITSU AMERICA Bob Driscal Kohji Mohri GIGABIT LOGIC Carl Deierling Jack Guedj HEWLETT PACKARD LABS. R. D. Crawford Joey Doernberg Hans A. Wiggers Chu Yen IBM Henry Brandt Joseph R. Mathis Horst L Truestedt INTEGRATED PHOTONICS Michael Pugh LAWRENCE LIVERMORE NATIONAL LABS. Paul Rupert LOS ALAMOS NATIONAL LAB Wally St. John Don Tolmie PCO Jim Goell SIEMENS Schelto Van Doorn SLAC (IEEE P1596) David B. Gustavson STORAGE TECHNOLOGY CORP. Roger Cummings Floyd Paurus SUN MICROSYSTEMS Robert N Snively SUPERCOMPUTER SYSTEMS INC. Leonard Veil TANDEM COMPUTERS Ken Kotyuk Jim Smith The meeting was chaired by the Chairman of the Working Group, Dal Allan of ENDL Consulting. Dal also distributed an Agenda, which is Attachment 1. The first presentation was made by Paul Scott of AMD on the subjects of 4B/5B coding and TAXIchips. A copy of Paul's presentation is Attachment 2. He began by presenting an overview of the 4B/5B coding scheme, and he noted that the TAXI is capable of operating in 8 bit, 9 bit and 10 bit modes. He referenced the 4B/5B patent held by Jim Hamstra from his days at Sperry, and noted that AMD had paid a license fee for use of the patent. Dal Allan asked if non-FDDI use was covered by this fee, and was told that the AMD lawyers said that the TAXIchip was covered for use in all applications. Paul then defined the possible dc offsets in the three coding schemes in the TAXI, and moved on to consider the TAXI input structure and possible error modes. He noted that there is no simple mechanism for defining a single bit error in a NRZI data stream, and that it is also difficult for a jitter-induced error to change two zero bits to two one bits. Paul stated that all errors in a serial channel are specifiable in terms of jitter, and noted that the FDDI specification defines three types of jitter, namely Duty Cycle Distortion, Data Dependent, and Random Jitter. The first two of these do not have low frequency components, so that the Phase-Locked Loop (PLL) that is involved in clock recovery does not see the jitter, and thus the impact of the jitter is only on the setup and hold times of the input latch (this was later stated to not be absolutely true as the phase comparator in the PLL is somewhat effected by jitter). Duty Cycle Distortion (DCD) was stated to result from component mismatches, threshold uncertainties and dc offset, and Paul defined the DCD tolerance of the TAXI to be half the bit time - or double that required by the FDDI spec. Mike Pugh of Integrated Photonics that a second order effect of dc offset is that the signal to noise ratio is degraded is the signal bottom is not flat, and Paul agreed that this was because the edges no longer looked like ramps. Mike concluded that dc offset could therefore be expressed as a power penalty with regards to the optical components. Data Dependent Jitter (DDJ) was stated to result from bandwidth- limited systems and non-ideal terminations. Dal Allan asked if the major problem with termination was high-frequency components, and Paul agreed and noted that the input of the typical receiver amplifier is not resistive but complex, and this presented a problem in terminating all spectral components properly. Mike Pugh noted that one of the reasons for the architecture that he had presented at the previous meeting was the problem in adequately terminating high speed serial interfaces. Again the tolerance of the TAXI was defined as up to half a bit time. Paul then moved on to give examples of errors induced by jitter and other mechanisms such as alias syncs. He made some comparison of the error detection performance of the 4B/5B code with the IBM 8B/10B code presented at the previous working group meeting. Paul stated that his comparison ignored the running disparity check that is possible with the 8B/10B code as he was not sure how it would be employed. Paul then summarized the presentation on the TAXI chip, and generated considerable controversy by stating that the dc offset implicit in the 4B/5B code "caused a minimal reduction in sensitivity for the optical interface". A typical architecture of a Fiber Channel interface was described, and several ways that it might be implemented using present or future TAXI products were suggested. Paul noted that the encoder and decoder in the TAXI occupied 80 and 160 gates and have a maximum fall-through time of 5 and 6 gate delays respectively. Paul then identified a number of future TAXI-family products as follows: NAME DATA RATE (Mb/s) LINK RATE (Mb/s) AVAILABILITY 7968/9-125 32-100 40-125 Now -175 100-140 125-175 12/89 -40 6.4-32 8-40 2Q/90 HS (TTL I/O) 160-200 200-250 4Q/90 HS (ECL I/O) 320-400 400-500 2Q/91 GaAs 800-1000 1000-1250 4Q/90 Paul stated that with the exception of the GaAs part all of the above products would be designed to work over both copper and optical data links. He also said that the majority of TAXI chips in the filed at present are operating over copper links and not optical ones. In conclusion, Paul noted that AMD had signed agreements with BT&D for optical link components and Vitesee for items related to GaAs. Mike Pugh noted that handling a data rate of 800-1000 Mbits/s in the GaAs chip could be difficult, and Paul assured him that the numbers given in the table above were publishable numbers and that the internal design targets were higher. There was then a considerable discussion about Paul's comparison of the error handling characteristics of the 4B/5B and 8B/10B codes. Paul agreed to provide detailed derivations of the values stated in the comparison, and these details are given in Attachment 3. Dal Allan extracted from the comparison the statements that the 8B/10B code was 100 times more likely to produce a command code, and 10 times more likely to produce an Alias Sync, than 4B/5B - and Henry Brandt of IBM was then asked to respond to these items at the next meeting. Don Tolmie of Los Alamos National Labs stated that 8B/10B seemed easier to implement, and Paul emphasized that the extra complexity in the TAXI is a result of having to handle the three modes. Mike Pugh agreed that the usefulness of the running disparity check is unknown at this time, and that the susceptibility to alias sync is a negative for 8B/10B, but he questioned if an alias sync would always be accepted in the middle of a frame - which would be required in order to substantiate the order of magnitude difference in the comparison. Mike also identified two key advantages to the 8B/10B code - namely that a one bit error always results in an error that is limited to a byte and the limited dc offset. The next presentation was made by Don Knudson of Digital Equipment Corporation (DEC), who proposed a scheme that uses a different 8B/10B code and incorporates forward error correction. A copy of Don's slides is Attachment 4. It should be noted that this was the first Fiber Channel meeting that Don had attended, and therefore his slide on goals represented only his view of what should be the goals of the Fiber Channel effort - which is a subset of the goals defined by the Fiber Channel SD3. Don defined the features of the proposed code as including a rich command alphabet and end to end synchronization. He gave a number of reasons for including forward error correction, including a tradeoff of CMOS logic complexity versus expensive, high-power laser transmitters. He cautioned of the presence of noise floors that cause the lower limit on the Bit Error rate (BER), and which cannot be overcome by increasing the transmitter power. Mike Pugh, however, cautioned that further work is necessary to quantify the noise floors for both Mode Partitioning Noise and Reflection- Induced Noise, and noted that the extra bits required by the forward correction scheme may significantly effect the Mode Partitioning noise floor. Don agreed, but stated that in the DEC situation (a dispersion limited system) the two effects balance. Don also noted that there is less control of the Reflection-Induced Noise, and stated that in some implementations in which already- installed cable plant is used a BER floor of 10-9 had been experienced. The forward error correcting scheme was then described in detail. It was noted that the error correcting bits were calculated on the basis of the encoded data, and that the correction was performed before decoding. This avoids the error multiplication effect in the decoding process. The 8B/10B code was described as having a maximum run length of 4 bits, and the encoder and decoder were said to be implemented in CMOS chips using table lookups. Don does not believe that CMOS complexity is an issue, and he stated that CMOS is usable even at Gigabit link speeds by using a 32 bit wide data path. He noted that the DEC patent also covers a 16B/20B code that provides perfect dc balance, but they do not see it as being significantly better in system terms than the 8B/10B code. Don then concluded with a description of a "brute force" implementation of HSC on a 1 Gb/s link. It was noted that this implementation only works where two HSCs are used in a full-duplex configuration such that the Ready can be multiplexed on to the opposite direction fiber. Also noted was the fact that parity is peeled off at the transmitting end and recreated at the receiving end of the link. Dal Allan noted that in the DEC scheme a 20% increase in laser bandwidth was traded off for a higher BER. Bob Snively of Sun Microsystems asked if the forward correction scheme took the place of packet checks and was told that checks at the packet level are still required. Mike Pugh of Integrated Photonics then reprised his presentation of a Fiber Channel Low Level Architecture from the last meeting. His slides from the last mailing are again included as Attachment 7. Paul Rupert of Lawrence Livermore Labs. then volunteered to provide a paper by NEC that included a discussion of the noise floors that had been referenced by Don Knudson, and this is included as Attachment 6. Monday afternoon then began with a presentation by Bob Carter of 3M on reflections in fiber optic systems. A copy of Bob's slides is Attachment 5. Bob began by defining reflections and methods of measuring them. He then identified causes of reflections - namely media boundaries, connectors, attenuators etc. He gave details and parameters for both fusion splicing and two sorts of mechanical splice manufactured by 3M. Dal Allan asked for details of the equipment necessary to perform fusion splicing, and was told that it is in the range $10K - $20K and includes a video system to aid in alignment (especially required for single mode fibers). It was noted, however, that this process is not normally used during fiber manufacturing as lengths of up to 25 KM can be produced. Don Tolmie asked why splices would be used instead of connectors, and was told that the splices are less expensive and easier to change as they use no epoxy. In answer to a question by Paul Rupert, Bob gave a price of $15 - $20 in quantity. Bob then provided some solutions to reflection problems in terms of ferruled connectors, positive contact end shapes and special end finishes. He described the telecom standard AT&T-developed contacting biconic connector and gave some statistical figures for return loss and attenuation derived from field experience. He described some of the difficulties of assembling the biconic connector and also referenced the ST and FC connectors, the latter of which has the highest market share in Europe. He also mentioned the SC connector, which is bayonet type created by Nippon Telephone and Telegraph (NTT), and which is quoted at half the price of an FC connector. He noted that 3M will be producing the SC connector by the end of 1989 and that NTT is working on a multiple version. The Dorran Biconic SPA connector was then described. This is a non- contacting connector in which two angled faces are used to minimize reflections. Bob also gave some statistical figures for the attenuation and return loss for this connector and noted that the SPA cannot be field mounted at this time. Bob provided handouts which included a reflections bibliography and details of the advantages of positive contact finishes, zirconia ferrules, and the SPA connector. These have also been included in Attachment 5. Bob was followed by Jim Goell of PCO, who presented his perspective on the Fiber Channel objectives, and upon the technology available to achieve them. A copy of Jim's slides is Attachment 8. Jim began with a list of the major issues and natural data rate breakpoints and the latter, developed independently, showed remarkable agreement with the breakpoints identified in a working group meeting over six months ago. He then moved on to identify the transmitter and receiver requirements of a number of current systems. He listed some current PCO products, and estimated prices of $400/pair of 200 Mbit modules with ECL or TTL interfaces in modest volumes. He stated that he expected modules for use with plastic fibers at 50 Mbits to cost $200/pair. He also stated that in the next 6 months PCO will have a laser transmitter in the same package, and that in two years he expected it to cost the same as today's led prices. A number of candidate physical layers and switching implementations were then described. Roger Cummings asked for an idea of the operating distances of the layers, and Jim replied as follows (noting that all the layers used 1300 nm components): LAYER DISTANCE 1250 Mb/s 5 - 10 KM (limited by mode partition noise), 200 Mb/s laser 25 - 50 KM 200 Mb/s led 2 - 3 KM 50 Mb/s led 5 - 10 KM Roger Cummings then lead an effort to obtain volunteers to begin determining the parameters of the various components that need to be considered in the systems design of the various classes of the Fiber Channel. The classes identified were: Coaxial Class 200 Mbits/s @ 10 meters Plastic Class 5 - 10 Mbits/s @ 20 meters Multimode LED Class 200 Mbits/s @ 2 Kilometers Multimode Laser Class 200 Mbits/s @ 5 Kilometers OR 1000 Mbits/s @ 750 meters Single Mode Class 1250 Mbits/s @ 25 Kilometers The persons who volunteered are identified in the Action Item list at the end of the minutes along with the components for which they were actioned. Tuesday morning began with a report by Don Tolmie of an ad-hoc meeting that had been held at Los Alamos during the week of July 10. The purpose of the meeting was twofold - to determine how to carry data at HSC speeds across the continent, and to investigate use of VLSI being developed for SONET. With regards to the first purpose, Don stated that it had been determined that only SONET supported the data rates required. Therefore making a gateway to SONET is the recommended approach, using a type of window protocol. With regard to using SONET VLSI components for the Fiber Channel, it was determined that a number of the SONET features would complicate such a use and lead to unacceptable performance in the areas of obtaining synchronization, latency and overhead. Dal Allan commented that these features are related to the billing structure and are thus not relevant to the Fiber Channel. Wayne Roiger of Cray Research asked if the Fiber Channel could make use of the optical devices and components being specified for SONET. Don replied that the components are mostly defined for longhaul applications and that there is a specification issue as SONET links are defined at a mod-split and not at the connectors. However an offer of help in the definition of the Fiber Channel has been made. The meeting then moved on to consider skeleton draft standards for the FC-0, FC-1, FC-2 and FC-3 layers of the Fiber Channel that had been produced by Don Tolmie. Both Mike Pugh and Henry Brandt had a problem with mentioning a copper version in these documents on the grounds of dilution of effort and a potential loss of volume for the optical version, but it was agreed to retain the references for the present. It was also reaffirmed that the intention is to multiplex SCSI, IPI and HSC devices on the same channel. Updated versions of the three skeleton draft standards will be distributed at a future working meeting. However, for information only, versions with rough markups of the changes agreed are included as Attachments 11 through 14. Paul Rupert of Lawrence Livermore Labs. then presented the results of some testing that had been performed by Ancor Communications on three types of optical transmitter. A copy of Paul's presentation is Attachment 9. He emphasized that the test setup did not mode fill the fiber. Mike Pugh enquired about the ease of modulating the 800 nm CD laser and was told that it wa satisfactory. Mike also raised the issue of launch ability and connector cost, and it was agreed that the connection cost would be equivalent to the cost of the laser. Jim Goell stated that there is little difference in chip cost between 800nm and 1300nm lasers. Paul then moved on to describe the HSC-based switching system that is being created at Lawrence Livermore Labs. A copy of his slides is Attachment 10. He noted that the system is based on a Klaus switch rather than the store and forward system being created at Los Alamos, and stated that it is intended to serve 2000 users distributed over a square mile. Dal Allan then reprised some of the agreements from the March working group meeting at San Raphael, CA with regards to the protocol of the fiber channel. He noted that all fields had been agreed to be 64 bit aligned and that an addressing width of 64 bits and the FC-2 level and 16 bits at the FC-1 level had been defined. He also defined some of the header fields. ACTION ITEMS 1) John Severyn of Lawrence Livermore Labs. to produce a matrix comparing the features of the FDDI 4B/5B, IBM 8B/10B, and Naked coding schemes. 2) Mike Pugh to further define the parallel transceiver interface in his Low Level Architecture. 3) Paul Scott to produce a definition of Bit Error Rate. 4) Roger Cummings to identify the requirements placed on the FC-2 protocol by the IPI Device Generic protocols. 5) Bob Snively to identify the requirements placed on the FC- 2 protocol by the SCSI command protocol and message structure. 6) Don Tolmie to produce updates of his draft skeleton standards. 7) Henry Brandt of IBM to respond to the comparison of the error handling characteristics of the 4B/5B and 8B/10B codes. 8) Kevin Able of Corning to define the parameters for the media of both the Multimode classes (both 50 and 62.5) and the Single Mode class. 9) Chuck Brill of AMP to define the parameters for the media- end connectors of the Coaxial, Multimode and Single Mode classes. 10) Bob Carter to define the parameters for the patch panels of both the Multimode classes and the Single Mode class. 11) Paul Scott to define the parameters for the media and transceivers of the Coaxial class. 12) Schelto Van Doorn of Siemens to define the parameters for the transceivers for the Multimode LED class. 13) Ron Soderstrom of IBM to define the parameters for the transceivers for the Multimode Laser class. 14) Jim Goell to define the parameters for the transceivers for the Single Mode class. 15) Gary Labelle of Avantek to try to get a representative of Hewlett-Packard experienced in their plastic fiber system to take responsibility for defining the components for the Plastic Class.