June 8, 1989 TO: X3T9.3 Fiber Optic Study Group Members FROM: Roger Cummings SUBJECT: FIBER CHANNEL WORKING GROUP MINUTES Please find attached a draft of the minutes of the ANSI X3T9.3 Fiber Channel Working Group of May 29 and 30, 1989. Note that there are also eleven Attachments to the minutes that relate to presentations at the meeting. Given the continued growth in the working group membership list, a review of the mailing policy has been made. As a result the concept of primary contacts has been abandoned, and the package of minutes and attachments will be directly mailed only to those persons who have attended at least one of the two most recent working group meetings. Note that the full package of minutes and attachments will continue to be included in the regular X3T9.3 mailing that results from the plenary meeting following the working group. Thus interested parties that are unable to attend the working group meetings are strongly advised to subscribe to that mailing. If there are any corrections required to, or omissions noted from, the minutes I can be reached as follows: Phone: (303) 673-6357 (Business) Telex/MCI Mail: (650) 289-5060 Fax: (303) 673-5891 Regards Roger Cummings Senior Engineer Subsystems Controller Development #nah0/rc MINUTES OF THE SEVENTH FIBER OPTIC WORKING GROUP MEETING The Seventh meeting of the ANSI X3T9.3 Fiber Optic Working Group was hosted by John Renwick of Cray Research at the City Center Marriott in Minneapolis, MN on May 22 and 23, 1989. A total of 41 people attended, as follows: AMD Jim Kubinec Alan Wendler AMDAHL Ed Cardinal AMP Edward F. Mikoski Jr. ANCOR COMMUNICATION Terry Anderson Rob Benton AT&T MICROELECTRONICS John H. Kemps CDC Lee Hartung Wayne Sanderson CONVEX COMPUTER CORPORATION Thomas W. McClendon Gary Stager CRAY RESEARCH INC. John Renwick Wayne Roiger DONAVAN INTERNATIONAL Don Pederson E. D. S. RESEARCH Ed Krohne FORD AEROSPACE Gary Waldeck FUJITSU AMERICA Bob Driscal Kohji Mohri GIGABIT LOGIC Carl Deierling HONEYWELL SSPL Jerry Quam IBIS SYSTEMS Bob Therien IBM Henry Brandt John Disbrow Gerry Heiling Ken Meifert Ron Soderstrom Horst L Truestedt IBM COMMS. Albert Widmer INTEGRATED PHOTONICS Michael Pugh KENDALL SQUARE RESEARCH Ed Gershenson LAWRENCE LIVERMORE NATIONAL LABS. George Bush John Severyn LOS ALAMOS NATIONAL LAB Richard Thomsen Don Tolmie NCR John Lohmeyer NETWORK SYSTEMS Ken Drewlo PRISMA Mike Baird SHELL DEVELOPMENT CO. Patric Savage SIEMENS Schelto Van Doorn STORAGE TECHNOLOGY CORP. Roger Cummings SUPERCOMPUTER SYSTEMS INC. Leonard Veil The meeting was chaired by Don Tolmie of Los Alamos National Labs in the absence of the regular chairman, Dal Allan of ENDL Consulting. Dal had, however, provided an agenda for the meeting, and this Agenda is Attachment 1. Don opened the meeting by reviewing the agenda, and noting that Kumar Malavalli of Canstar was again not in attendance. The information was also volunteered that Howie Johnson, the scheduled presenter from Ultra, was no longer with that company. Don also distributed notices for the June and August X3T9.3 plenary weeks, which are Attachments 2 and 3 respectively. John Renwick distributed a list of activities in Minneapolis for the week, which is Attachment 4. Wayne Sanderson of Control Data distributed the minutes of the last Fiber Channel Working Group Meeting which, because they have not been distributed other than at the meeting, are included in these minutes as Attachment 5. The business of the meeting the began with a short presentation by Schelto Van Doorn of Siemens. Schelto explained why Siemens, which presently produces surface-emitting light-emitting diodes (SLEDS), intended to move directly to Lasers and not produce Edge-emitting leds (ELEDS). He noted that ELEDs suffered from temperature instability which required a Peltier element to be included which was less reliable than the led itself. He did however note that ELEDs had the advantage of narrow spectral width. Schelto was questioned about the possible modulation rate, and stated that 200 MBaud was about the limit. He also described low cost laser dubbed the "Volkslaser" that Siemens has on the drawing board and that would not require a Peltier element. Mike Pugh of Integrated Photonics noted that the structures of ELEDS and Lasers are so similar that it is a major problem to stop ELEDs lasing and thus causing the back facet to fail. Albert Widmar of IBM Yorktown then presented IBM's 8B/10B coding scheme. A copy of Albert's slides is Attachment 6. He began with a description of interface functions and contrasted the features of typical computer and common carrier links. He saw the major difference as one of optimization - in a computer link the throughput is dependent upon the electronics where a common carrier link is designed to go as fast as the transceivers allow. He then gave a brief introduction to transmission codes and their characteristics, beginning with the simple Manchester system. He noted that although ternary codes would be very efficient, they had never been used due to the problems in defining the thresholds in a ternary receiver. The major features of the 8B/10B code were then described as being its binary nature, the fact that it is dc balanced, its limited run length, and that the encoder and decoder can be relatively simple combinatorial circuits and not table lookups. The fact that the code is partitioned into 5B/6B and 3B/4B sub-blocks was noted. He stated that the code detected errors better than simple parity and noted that complementary 1 and 0 errors in close proximity were the most difficult errors to detect. Albert described the term "Disparity' as the difference in the number of ones and zeros that had been transmitted, and he showed an envelope diagram that demonstrated that at the end of each six bit time the disparity would be plus or minus one. He then compared a number of coding schemes in terms of the availability of symbols of limited disparity. He noted that the 4B/5B code provided 16 code pairs but that it was very difficult to fully balance the code. He emphasized that disparity was not as important as the long term dc offset. The 10B alphabet was then described in detail as consisting of 256 data characters, 3 comma characters and 9 other special characters. The constraints of the code are a maximum run length of 5 and a maximum digital sum variation of six. With random data the code provides 60 transitions per 100 Baud and the worst case is 30 transitions per 100 baud. The comma characters are used to uniquely define byte boundaries, and thus can be used for instantaneous byte synchronization. In this code the comma characters consists of a run length of two followed by one of five, and thus are easily detected. The other special characters are available for use by the protocol. More special characters can be defined, but at the price of increasing the complexity of the encoder and decoder. Albert then provided some implementation details of an 8B/10B encoder and decoder. He noted that because of their simple structure they can operate on the parallel side of the serdes chip, and thus at the byte rate. The first implementation used 130 MECL 10K gates with an additional 40 gates being added for the purposes of error detection. The gates were mostly logic but with some flip- flops. There is, however, already an implementation in Gallium Arsenide. Albert showed a micrograph of the receiver chip, which measured 3mm x 4mm and contained four integrated receivers including photodetector and clock recovery. He stated that the short run lengths of the code make clock recovery relatively simple, and that the dc balance allowed a simple capacitively coupled receiver design. Wayne Sanderson of Control Data inquired about products based on 8B/10B code, and was told that Hitachi have had a channel that uses this code for two years. Don Tolmie asked about patents on the code, and Henry Brandt of IBM Kingston replied that, although no definitive position had been established, he did not anticipate any problems because of the cross licenses that already existed. Jim Kubinec noted that the encoder/decoder scheme was not purely combinatorial, and Albert agreed that a flip-flop was needed to hold the running disparity in the encoder, and that one could be used in the receiver for error detection purposes. Mike Pugh emphasized that the simplicity of the clock recovery scheme in the 8B/10B decoder would very much simplify the task of building a monolithic implementation due to the limited Q factor required. Jim Kubinec of AMD asked if the code characters had been examined for the possibility of aliasing (i.e. boundary shifting), and was told that this had not specifically been investigated, but that many errors would result from such a condition. He further stated that a maximum of five bit errors would result from a single bit error in the encoded data. There was then a short discussion about the error characteristics of Fiber Optic links. It was noted that Gaussian characteristics would normally be expected, but that a non-ideal threshold and external factors such as ESD would cause modification. Mike Pugh noted that at the noise limit (which is what defines the Bit Error Rate) lasers generate more noise when operating than not, and thus more logic "1" errors would be expected. He also noted that there is nothing inherent that causes an error to propagate into an adjacent bit cell as there is for example in disk media. Integrated Photonics field experience has indicated that in a well-designed fiber link problems with power supplies and ground shifts are the major causes of errors. A comparison to 4B/5B was again requested, and Albert noted that because the code is not dc-balanced receiver sensitivity is lost, and the receiver is more difficult to design. Schelto Van Doorn stated that one of the reasons that 4B/5B was chosen was that it was thought to use less silicon. Jim Kubinec stated that at the time of the decision it was expected that nibble-wide interfaces would have some attraction, but that had turned out not to be the case. Terry Anderson then gave a presentation on "Naked Coding" - a term that at the outset Terry stated that he was not sure he understood. It was suggested that this term had originated with SONET, and Terry moved on to describe in detail a proposed framing structure for the Fiber Channel that had been derived by Ancor Communications from their present products. A diagram of the framing structure can be found on page 5 of Attachment 10. Terry noted that the major requirement for Ancor's products was a very low error rate (a BER of 10-15) in the delivery of the payload. The means for achieving this was the use of a Hamming code which was able to provide for correction of single bit errors and the detection of double and some other multiple bit errors. He noted that 4B/5B requires 40 bits to encode a 32 bit data word, and that the Hamming code can be fitted into this space. Terry then moved on to consider the fields of the frame in detail. Ancor's current products use a structure with a fixed 256 word data burst. However the frame structure that he now proposed included a data burst that could vary from one to 256 words. Each 40 bit word contained four bytes of data, seven check bits and a transition bit whose state was varied to aid in maintaining a dc balance. The check bits were positioned after every 5 data bits to aid in maintaining transition density. A two word "header" preceded the data burst and contained length information, burst identification and a sync field. A one word Longitudinal Redundancy Check (LRC) followed the data burst. Terry was asked for the maximum run length defined by this scheme and responded that it was 10 bits. Patric Savage of Shell Development asked how this could be guaranteed given that the LRC word was not controlled. Terry responded the 10 bit figure was proven by Ancor's experience and that fiber component manufacturers were not generally concerned about run lengths until they reached the order of 60 bits. After some discussion it was agreed that although the maximum run length was clearly deterministic it would not be simple to prove theoretically. Mike Pugh noted that some links use a scrambler in place of a coding scheme such as 8B/10B and that in that case a statistical model can be rigorously applied. Don Tolmie asked how dc balance was maintained and Terry reiterated that a running disparity was calculated and used to control the transition bits in each word. He noted that the variable length data burst would mean that each frame would not necessarily be dc- balanced. Mike Pugh again described the complex interaction of fiber receiver parameters. He emphasized that a longer run length meant that a higher Q factor was required in the clock recovery circuit and that a monolithic clock recovery circuit would only be able to have a limited Q. He stated that other clock recovery schemes would be available at a reasonable cost and with a higher Q such as a surface acoustic wave device, but obviously without the advantages of integration. He also noted that dc offset determined the time constants in the determination of the receiver threshold. In the first presentation after lunch Mike Pugh described the experience of Tacan Inc. (Integrated Photonic's parent) with Fiber Optics. A copy of Mike's slides is Attachment 7. Tacan's experience encompassed three areas with the first being Cable TV systems. In these systems 40 channels are amplitude modulated on to a single fiber - which requires very linear systems and thus these systems use high cost lasers. The second area was fiber local loop prototypes, but Mike stated that Tacan did not see "fiber to the home" happening anytime soon given the ongoing quest for services to fill the bandwidth available with ISDN. Thus they did not see such "consumer" areas causing any huge increase in the demand for fiber optic transceivers. The third area of experience was Integrated Photonics' Toplinc and Topnode products. Toplinc is a 1300 nm product that can operate over a distance of 2 Km using multimode fiber (including a budget for four inline connectors) and 30 Km using single mode fiber. It operates at 500 Megabaud and up to six units can be paralleled to give a 3 Gigabyte/s bandwidth. Toplinc uses scrambling in place of a coding scheme - which is appropriate as it has a point to point architecture. Mike pointed out that it would however not be appropriate for a switched architecture because of the need to reestablish framing. Topnode is a variation of the Toplinc product designed to act as an HSC extender (it claims to be the first HSC product). It encodes both the HSC data and control signals on to the fiber using a "brute force" approach (i.e. no attempt is made to take advantage of the lower repetition rate of some of the control signals). Mike stated that Topnode is not being proposed for standardization as it uses older technology that requires three fibers where with later technology one would suffice. Mike then moved on to propose an architecture for the Fiber Channel - at least at the Physical level. But first he considered where the major costs of a fiber channel will reside when a high volume has been attained. His conclusion was that the major costs will be for materials and the packaging capable of handling both the high frequency signals and the dissipation required. Mike then listed the basic elements of a Fiber Channel System Architecture. and then considered how these elements were grouped in both Integrated Photonics' Toplinc and Gazelle's proposed Hotrod product. In Toplinc the partitioning was defined primarily to minimize the number of high frequency signals that had be routed on the circuit card. Mike stated that many Toplinc customers had express concern at having to handle 1 Gigabit differential ECL signals on a standard technology card. In the Hotrod scheme all of the elements with the exception of the FO Transmitter a Receiver are integrated into the single GaAs chip. The number of pins (200+), the high speed of some of the signals, and the dissipation of the chip make packaging it a very expensive proposition. On consideration, Mike had come to the conclusion that neither of these architectures was appropriate for the Fiber Channel. The cost of the Hotrod chip would make a low end application almost impossible, and the Toplinc architecture would entail layout complications that would be difficult to specify. Therefore Mike's favored architecture for the Fiber Channel was one in which all of the high speed elements resided in one transceiver chip, or in separate transmitter and receiver chips as shown in the slides. Each side would then communicate via a parallel interface with the other elements contained in a single chip. This chip would then be able to be fabricated in a standard CMOS or CMOS/BIMOS process, and contained in a standard package. Mike had gone so far as to define an implementation using this architecture. This used either a Class 1 laser or a Class 3B 1 mW laser with 0 dB launch power at 1300 nM. This would be connected to a single mode fiber using a FC connector (Mike noted in passing that British Telecom and Dupont have a laser in an FC-connectorized package for $270 today). The receiver would be a pin photodiode and the interfaces from the Transmitter and receiver chips to the main cmos chip would be 10 bit wide ECL interfaces. Mike stated that and ECL interface had been chosen over a TTL one because ECL generated less noise at the cost of a termination. Schelto Van Doorn noted that the CMOS chip still has a high pinout and Mike agreed but emphasized that it is a standard chip with no high frequency components. He also pointed out that if the same chip could be used for all Fiber Channel classes then it could easily become a merchant item. However presently less than 100,000 laser are produced each year for the communication market so the Fiber Channel would have to create a demand at least an order of magnitude higher for this to become true. John Lohmeyer of NCR pointed out that for one CMOS chip to be used in all situations then the same parallel ECL interface would have to be used. Don Tolmie asked why it would be necessary to standardize on that parallel interface when the goal of the effort was interoperability on the fiber. Mike gave an excellent reason in return, namely that the CMOS chip and the high frequency transmitter and receiver chips would normally made by very different organizations. Thus a standard for the interface between them would very much aid all three parts becoming merchant items. John Lohmeyer noted that there was a distinct precedent for this in the standardization of the Ethernet transceiver interface. Schelto Van Doorn emphasized that if merchant parts were desired that the functionality should be kept general. Mike Pugh noted that designing a generic clock recovery scheme would be a challenge but that he thought that it could be achieved. Jim Kubinec stated that the FDDI cost reduction effort is considering just this sort of repartitioning. Ron Soderstrom of IBM Rochester noted that this architecture supports multiple physical classes very well, and John Kemps of AT&T Comms noted that the major reason why nothing standard is available in this area is that it has proved impossible to get multiple vendors to agree to a common partitioning of functions or even a common coding scheme. Roger Cummings asked if the same coding scheme was applicable to both a fiber and a serial copper link. Mike replied that it is providing that the bandwidth is adequate, i.e. that adjacent bit and frequency attenuation problems are not encountered. He also noted fiber optic systems are normally limited by attenuation rather than dispersion. Schelto Van Doorn then gave a brief reprise of his presentation at the June 1988 Working Group meeting. This was on the subject of the array produced by Siemens' medical division for use in a fiber optic link. A copy of Schelto's presentation is Attachment 8. Schelto was followed by Edward Mikoski of AMP, whose subject was connector requirements and liaison with EIA. Before Ed began Don Tolmie defined the guidelines for the liaison, which is that the group will attempt to abide by EIA connector guidelines and to cooperate with the appropriate EIA groups. However the control of the effort will remain with X3T9.3 if no good, timely response is received. Ed began by introducing himself as the chairman of the EIA FO-6 committee, which is chartered in the area of fiber optic components. FO-2 is the equivalent committee on the systems side. FO-6 has developed 180 fiber optic test procedures, of which 100 have been published. There has apparently been DoD involvement in 60 procedures. Ed had come equipped with a detailed checklist to establish the required budget figures and connector features. A copy of the checklist is Attachment 9. Although Ed obtained few hard requirements in response to his checklist, some clear sentiments did appear. Nobody expected that the Fiber Channel would work with fiber that already exists within buildings. A strong sentiment in favor of the use of an existing connector was evidenced, and 500 lifetime termination cycles was agreed as a preliminary requirement. Some people were surprised to learn that most copper connectors are also specified at this level. Using a standard ferrule for all FO classes regardless of the connector type was also suggested. A strong feeling against the use of index matching gel was also expressed. The requirement for field termination was less clear, as a number of people pointed out that users do not generally terminate copper cables today. In particular the MAC connector was noted to be not heap and not able to be field terminated. Mike Pugh noted that their new product included budget for 10 inline connectors, as opposed to four or the previous generation, and that had been well received by prospective users. The issue of transmitter and receiver spacing also generated some debate. It seems that there is a minimum required spacing between transmitter and receiver to avoid interference. Terry Anderson noted that Ancor mounts its transmitters and receivers on different sides of the board. Schelto Van Doorn suggested the use of pigtails or direct connect to meet tight space requirements. The question regarding fiber type elicited the fact that both Ancor and Integrated Photonics use 50 um fiber in their latest products. The 50 um fiber has superior transmission characteristics but presents a much more difficult launch problem, and this is especially significant for leds. The final presentation on Monday was made by Terry Anderson, who presented a Fiber Channel Standard Proposal that incorporated the framing scheme that he had described earlier. A copy of the proposal is Attachment 10. The proposal emphasized "keep it simple" and incorporated seven classes starting at 250 Mbits/sec. It suggested the use of 1300 nm components with leds, 50/125 um mm fiber and "ST" type connectors for distances of less than a kilometer, and lasers, 9/125 um sm fiber and AT@T "Biconic" type connectors for distances of greater than a kilometer. A number of comments were made that 250 Mbits/s is too high for the basic Fiber Channel speed, and 50 Mbits/s was substituted. A sentence indicating optimization for 4 Megabyte block transfers was also deleted. In answer to a question Terry noted that he expected a 250 Mbit/s transceiver pair to cost "way under $100 in volume in "ST" type packaging. John Lohmeyer stated that this would be significant in a SCSI marketplace that is 80% single-ended and 20% differential and where the significant delta for differential is not the $20 cost but the extra board area required. Jim Kubinec noted that AT&T have a cmos clock separator product that operates at up to 200 Mbit/sec. Terry noted that Ancor's cmos vlsi chip that was described at a previous meeting operates at 250 Mbits/sec and costs $53 in quantity. Tuesday morning began with a presentation on the security of high speed channels by George Bush of the Lawrence Livermore Laboratory. A copy of his slides is Attachment 11. George began by stating the fundamental security principle of the control of access to information, and given that the only truly secure system is one that is powered off, he viewed the security issue as "being between a rock and a hard place". George classified the main risks as being the modification, disclosure and destruction of information along with denial of access to that information. He stated that the much publicized hackers were the tip of the iceberg - with most security problems resulting from simple mistakes, environmental hazards (flood etc.) and disgruntled employees. George then moved on to consider Physical Security issues. He noted that bus systems rely more on physical security than the networks that they replaced, and stated that encryption replaced only part of physical security. In response to a question he stated that today encryption operates at 50 Mbits/sec maximum. He then reviewed authentication techniques noting that hand geometry, although well established, was regarded as not very unique and that once promising techniques based on keyboarding patterns had proved to have many problems. George then provided a risk assessment of security measures, emphasized the problem of diminishing returns versus costs, and classified valuable and invaluable data (the latter being concerned with national security for example). George then moved on to what he regarded as the very key subject of security administration. He stated that "we all feel that we work with good people", noted that trust is not assurance and emphasized that the loop must be closed with verification. He also emphasized that physical security calls for very careful administration to be successful. George then closed with a consideration of security issues specific to high speed channels. He saw the major issue as being the concentration of facilities, with the disadvantage of being a single point of failure and thus an attractive target, but the advantage of being a single point of control that could facilitate centralized authentication. He gave the information that fiber could be tapped, given the ability to work in the wiring closet for half an hour and to bend the fiber. He stated that plastic fiber is worse because it can be cut with a razor blade and reconnected with standard epoxy. He described a system developed by Hughes Aircraft named IDOCS in which the fiber sheath is used as a lossy channel whose interruption is used to detect an intrusion and shut down the main channel. Schelto Van Doorn asked if sudden power changes could also be used to detect intrusions, and George replied that in practice he suspected those techniques had problems with sensitivity. John Kemps asked why encryption, even if it were possible at fiber optic channel speeds, would only replace part of physical security and George replied that the encrypters at the nodes would still have to be physically secure. Ken Meifert of IBM then reprised his earlier presentation of a scheme in which the HSC concepts were included in a proposed framing scheme for the Fiber Channel. This lead to a discussion of possible framing errors in which a false sync is established on a part of the data field. It was noted that some systems rely on coding violations to establish frame sync e.g in Manchester systems. Again it was emphasized that the IBM 8B/10B code has special characters for synchronization, and that more than a simple error is required for one of these characters to appear falsely. This led Albert Widmer to give a detailed probability analysis of the error detecting capabilities of the 8B/10B code which again showed its superiority to simple parity. The slides that Albert used were merged with his earlier presentation for convenience, and thus can be found at the end of Attachment 6. Don Tolmie asked for a comparison of the 8B/10B and the ECC scheme proposed by Terry Anderson. It was agreed that in a well-designed link that single-bit errors would dominate and a rule of thumb that 1dB of receiver sensitivity results in three orders of magnitude improvement in the Bit Error Rate was quoted. It was agreed that the BER and undetected error rate were the parameters of most concern to fiber link users. John Kemps noted that as a link will never be dark after establishment, then ECC could be performed on a time basis as well as a frame basis. The conclusion of the discussion was that the two schemes would be comparable in implementation, and a number of ways of combining them were discussed with no clear conclusion. It was clear, however, that the propagation of a single bit error on the channel into multiple bit error at the decoder output would put stringent requirements on an ECC code. Patric Savage asked that single bit error correction be included in the Fiber Channel requirements, and this touched off an extended discussion which lead to Don Tolmie proposing the following motion: "That the Fiber Channel will feature error detection only until a justification and an implementation proposal for error correction is presented to, and approved by, the Working Group." This motion passed unanimously - although it was noted that the persons who had argued most strenuously for error correction had left the meeting by the time that the motion was presented. A further motion was proposed by Don Tolmie: "That the 8B/10B coding scheme, as proposed by IBM, be adopted for use in the Fiber Channel - assuming that ANSI patent requirements can be met." The votes on this motion were For 18, Against 1, Abstain 1. The objections seemed to be made on the basis that the commitment to a coding scheme was premature. The following action items were defined for the next meeting: 1) Henry Brandt to clarify the patent position with regard to the 8B/10B code. 2) Kumar Malavalli to present a broadcast hub configuration, and clarify the position with regard to Canstar patents. In addition it is intended that at the next meeting the Ancor proposal will be further discussed, and that work on the adaption of the IPI and SCSI protocols will begin.