Home >  Articles > ATM testing and assurance

•  
•  
•  
•  

Asynchronous Transfer Mode (ATM) has become an essential part of just about every carrier's backbone infrastructure and has significant potential to be a key component in the next generation of LANs.

The promise of ATM, which is based on its high speeds, efficient cell switching, and simultaneous support for multiple Classes of Service, can only be fully realised if ATM-based networks work according to the published specifications. This is not as simple as it may seem at first glance. Two of its key benefits, high speed and multiclass support, are often at odds because the Quality of Service (QoS) expected for the various classes of service becomes increasingly difficult to manage and implement as traffic loads increase and speeds accelerate.

Implementations of complex mixes of service classes and their related QoS parameters, which are so necessary for effective ATM performance, may not always work as advertised. In order to properly implement ATM systems, manufacturers and developers need to put their products through extensive testing and fine-tuning to be sure that they work properly in the face of highly varied and challenging traffic loads. Carriers implementing ATM on a day-to-day basis need to assure themselves that each customer is being served properly and in accordance with the negotiated QoS they expect.

All successful networks need to be reliable but ATM, given its high capacities and complex features, is especially sensitive to service degradation and failure. One of the best ways to guarantee long-term performance is through rigorous, standardised testing at both the component and system levels.

Testing systems check whether ATM equipment meets design specifications, confirm that products from different manufacturers can work together, measure performance against traffic contracts and provide assurances that the underlying physical resources are operating correctly. However, the complexity of large scale ATM network infrastructures, the advanced software mechanisms used by ATM systems, and the variable nature of multimedia traffic all combine to make testing a non-trivial exercise. ATM does not exist in a vacuum but serves as a switching and transmission vehicle for all types of higher-level network traffic. Testing systems, therefore, have to include the ability to examine different, traffic related, higher protocol layer issues such as the mapping of connection-less IP traffic onto the connection-oriented structure of ATM. The varied protocol characteristics of IP, ethernet, SNA and other data supporting protocols must successfully work over ATM. Testing this capability is essential. Furthermore, ATM has to interwork with the logical structures (topologies) of LANs and WANs using varieties of connection and connection-less oriented transmission mechanisms. This demands that the testing platform be able to examine protocol mapping and translation functions as well as performance and throughput.

On top of this set of functions, the testing platform must be able to perform testing of the associated signalling protocols in order to establish end-to-end connections and to perform QoS testing.

Testing must be an essential element at every stage of an ATM network's life including the design and development phases as well as the deployment and operational phases. Testing that can be done at the function or module level during the design and development process reduces product time-to-market and increases product quality.

Similarly, ATM testing during deployment and operations, which must go well beyond basic functionality if QoS is to be controlled and managed, results in consistently high levels of customer satisfaction.

Three basic types of testing

As characterised by the ATM Forum, there are three types of critical ATM testing: Conformance, Performance and Interoperability. These are used to establish and validate acceptable levels of confidence that the tested ATM products meet industry requirements. However, each of these types of tests can be quite rigorous and costly.

The three types of testing can be performed independently and they are not ordinarily prerequisites for each other. Furthermore, the success or failure of one test is not a predictive of the success or failure of the other two. The best way to develop high levels of confidence is to combine all three allows for each ATM product set.

Conformance testing

ATM conformance testing is used to determine if ATM components meet the rules set by one or more standards, including both data transport and switched connection signalling. It consists of verifying that an ATM product meets the specifications as defined by either the ATM Forum or any official standards organisation. Conformance testing is also indicated whenever an interoperability problems arises between different pieces of ATM equipment to aid in the diagnostic process. Conformance testing is ordinarily quite extensive and consists of testing each possible ATM implementation against every feature and function that is defined in the specification.

Interoperability testing

Interoperability testing checks a product that has successfully passed conformance tests against other products that also claim to be conformant, using the test system to analyse the correctness of the operation. The goal is to ensure that interoperability works in practice as well as in theory through observation and measurement. Testing is especially needed for new implementations, for regression tests (when software or hardware is changed), and for longevity (long-term operation) testing.

Interoperability testing provides proof that equipment from different manufacturers (or even different models from the same manufacturer) can work together successfully.

Performance testing

Performance testing attempts to determine how well the product or service does its job in a specific operating environment after interoperability testing has been completed. Performance tests focus on measuring the QoS characteristics that are associated with both the data transfer and the signalling capabilities of an ATM network. For example, testing of signalling performance would determine the call establishment rate that can be sustained by a switch in a specific network. Traffic Management Performance test details will depend on the class of service being tested, on traffic conditions and on the specific traffic management policies that have been implemented in the switches.

Performance testing verifies the operational characteristics of data transfer and connection signalling in the network, while either in or out of service. It evaluates various implementations under different traffic types and volumes in order to measure compliance with pre-established performance parameters. Standardised performance testing is intended to define a common methodology for conducting performance measurements. In this context, performance parameters are related to specific ATM implementations, are functionally defined, and deal with three mutually exclusive characteristics:

• Speed (eg, Call establishment latency);
• Accuracy (eg, Connection setup error probability; and,
• Dependability (eg, Connection setup denial probability).

Load categories

Performance testing ordinarily deals with two categories:

• Measurement under Normal Load Conditions in which the test traffic matches the traffic contract and does not exceed system resources. Testing is conducted while the SUT is both in and out of service. Performance measurements include ATM layer QoS parameters and ATM Signalling including: cell transfer delay (CTD); cell delay variation (CDV); cell loss ratio (CLR); and throughput. Signalling statistics (call establishment latency, call release latency, call setup rate, etc).
• Measurements under Overload or Outage Conditions in which the test traffic exceeds the traffic contract or system resources. This is used to test the SUT congestion control and overload protection mechanisms such as the UPC (usage parameter control) and the NPC (network parameter control) for policing verification and at the signalling level to verify call establishment rate under maximum loads.

Tests can be categorised by the point in the network life cycle at which they are applied and for whom they are being done. Table 1 provides a cross-reference of test types of life cycle stage.

What must be tested?

Network testing has traditionally focused on proving that communications protocols are implemented correctly and are functioning as planned. With ATM, operational aspects such as QoS (performance achieved), SVC signalling, security, and OAM management are also subject to verification. Tests may also be required for the conversion between legacy networks such as ethernet or frame relay and ATM formats.

The protocol architecture for an ATM-based communication system can be divided into multiple layers and planes. Testing may be required for the protocols and specifications associated with each of the layers defined for each of the planes in this architecture. Testing the interfaces between the different layers and the internal algorithms used in the equipment is also beneficial, especially during the product development phase.

An ATM test system must be able to test the individual network components as well as the complete physical and logical network. The physical network includes the basic transmission media, the network interface standards, the network protocols, the different feature sets, and the underlying quality of service. The logical network includes virtual connections, per QoS connection parameters, congestion avoidance, fault recovery mechanisms and management controls.

Testing is required for the following aspects of an ATM network:

• Physical interface testing, which is a basic requirement for any test setup. Physically connecting to a network interface point so that traffic can be monitored and/or inserted is a prerequisite to any other level of test. The performance of the physical media also determines how well the ATM network can reasonably be expected to work. Being able to insert physical layer errors and alarms is required to verify correct operation of networking devices.
• User plane testing, which covers all the functions that support the transfer of user data across the network in the form of cells. The User Plane includes the ATM Layer and the ATM Adaptation Layer. Testing these planes focuses on the cell layer interoperability between two devices as well as the performance at both the ATM layer and the AAL layer for providing transport of (eg, IP) packets over the network.
• Signalling plane testing includes the functions that establish, maintain and clear the switched virtual circuits on the network. Signalling can be divided into user-to-network and network-to-network components. Testing of the end-to-end signalling functionality and performance is especially important when multiple service providers are involved.
• Management plane testing includes everything concerning operations, administration, and maintenance. Interworking of management elements is especially critical when multiple service providers are involved and when service level agreements involve financial penalties for non-performance. Being able to thoroughly exercise the Interim Local Management Interface (ILMI) interface and OAM operations are critical for Management Plane testing.

Testing methodologies

ATM system tests can be performed through:

• Out-of-service testing during the production process as well as during acceptance testing and commissioning, or whenever a failure occurs that interrupts normal service;
• In-service testing by monitoring the live network and analysing changes in function or performance (as compared to another time or a reference specification); or
• In-service testing, during live operation, by inserting predefined test messages (such as QoS test cells), possibly over an unused channel or via network management channels, as well as adding additional traffic loading on the interface to characterise types of traffic performance degradation.

It is always preferable that tests be carried out in a non-disruptive manner. In general, testing should be a combined effort of external test systems and an embedded network management system.

In order to most closely match the user's view of the system, tests should be performed on an end-to-end basis between the UNI for the data source and the UNI for the destinations. This, however, may not always be feasible in which case hop-by-hop testing will be necessary.

Static testing

Conformance testing is the first step in the product-testing phase and is based on the methodology defined in ISO 9646. A questionnaire, called a Protocol Implementation Conformance Statement (PICS), provides the way to state what 'profile', or set of capabilities and options, the producer has selected in building the product. The PICS proforma provides a standardised checklist that can be used during product evaluation to determine a product's suitability for a particular network or application.

Dynamic testing

Once the possibility of compatibility has been confirmed, a dynamic conformance test needs to be performed to show that the product actually implements the standard correctly (as opposed to just saying it does). The test system becomes the 'reference implementation' and a specific suite of tests is performed on the IUT (implementation under test).

ATM traffic testing

The ability of an ATM network to support various classes and mixes of traffic can be quite difficult and time-consuming to test. ATM user data, signalling data, and network management data is more complex by its very nature. The ATM test system has to distinguish between frame-based and continuous data streams, identify the priority levels of the traffic, detect the use of protocol encapsulation, and perform full seven-layer decodes for a variety of protocol suites. Specifically, ATM traffic testing is deployed in order to evaluate the QoS performance of the ATM layer using the ITU I.356, I.35BA, and I.353 standards as adopted by the ATM Forum TM4.0. It is used to characterise switch architectures, as well as for verification of performance goals when installing new ATM switches and to identify physical defects, etc.

Older testing technology ordinarily used simple bit error rate tests (BERT) on sample ATM Cells. With the new ATM Forum/ITU test cell format, testing is more effectively done by a Cell Error Rate (CER) measurement, at full link speed. This allows the measurement of cell loss, cell error, cell mis-insertion, cell delay (min, max, average) and cell delay variation. The relationship between the network QoS that is actually achieved and the ultimate performance of an application can also be quite complex. A unit of data as seen by an application may bear little similarity to the small cells that are the unit of ATM data transfer and therefore measures of success may need to be quite different. Examples of the differences include:

• File transfer over a LAN, where loss of a single ATM cell could result in the retransmission of large amounts of data at the application layer;
• Voice transfer over ATM, in which loss of a single cell may not be important but excessive variations in cell transfer delay could result in voice conversations that keep breaking up; and,
• Video over ATM: cell loss results in distorted bands on the screen. There is a strong need for assurances that everything works correctly both initially, when the network is first installed, and over time as new software is installed, as long-term conditions change and as traffic loads expand.

Traffic generation

An ATM test system must also be capable of generating traffic of various types and to combine it with test cells in order to simulate network characteristics.

One of the challenges facing testing agents is how to generate comprehensive, meaningful traffic that truly emulates real life traffic patterns. The complexity of multiple virtual paths supporting a highly mixed bag of traffic types cannot be effectively tested by simply sending a single data stream consisting of canned, static test messages. This problem can be characterised in two ways. There is the issue of emulating simultaneous traffic paths from multiple channels and there is the companion issue of generating complex, meaningful and application-specific traffic streams. This requirement means that the test platform must support multichannel GCRA (Generic Cell Rate Algorithm) generation that complies with, or exceeds, the GCRA specified in the switch policing mechanism. It also means that the testing platform must accurately generate all bit rates with a strong definition of a traffic-mixing algorithm. It must also have the ability to insert QoS test cells in all transmitter traffic streams to verify QoS for each stream transmitted.

This combination of complex messages on multiple channels is the only way to properly test for QoS results. Furthermore, the testing platform must be able to generate these traffic flows at wire rates and high volumes sufficient to overload buffers. Without this, traffic management policies cannot be effectively exercised. In an appropriately configured testing platform, traffic can consist of multiple, foreground traffic component(s) containing QoS 0.191 test cells which track loss and delay of cells through the switch, and background traffic that can be used to load up the network. In this environment all measurements are performed simultaneously on each channel, which are tested for both loss and delay.

QoS support

The ability to efficiently combine traffic with different QoS characteristics into one uniform data stream is the major technological advantage offered by ATM. This traffic can be characterised in terms of the type of data to be transferred as driven by the needs of the user application, how the data is handled within the network and what service qualities are required of the underlying network facilities.

ATM networks support several specific classes of service. Each class of service is targeted towards a different application type. The inherent nature of these applications usually dictates which QoS parameters will be required on each of the network connections.

QoS parameter control

QoS testing is usually performed under out-of-service conditions. Testing the quality of ATM Layer service requires a test system that is capable of generating ATM traffic and then monitoring that traffic to make specific measurements. Traffic needs to be generated at full line rates to stress the network and beyond if overload tests are required. Traffic generation must simulate live conditions using a range of available VCs, with adjustable test cell transmission rates and a variety of background traffic conditions.

Use of the test cell defined by the ITU-T O.191 recommendation provides the most accurate method of calculating ATM layer performance parameters. The primary advantage of using a standard test cell is that measurements are interpreted in a consistent manner. For example, when the cell format is known the test system can distinguish between cell errors and cell mis-insertion.

The challenges of ATM network testing

The benefits of ATM are obvious. It enables a single, integrated, high-speed network that can be used across the entire network to support diverse applications and traffic types at guaranteed levels of service. But these benefits are not easy to achieve.

One of the core concepts of ATM is that the limited resource of the network is used in a statistically multiplexed manner, balancing highly demanding traffic loads at the edge of the network's useable capacity. As already stated, complex, high speed traffic policing and management techniques are needed to make this dizzying trick work successfully. Not only is there the need to support competing traffic sources in a fair manner, but the traffic comes in widely divergent types. Voice traffic, connectionless internet, ethernet, frame relay and video all have significantly different profiles, demands and priorities. ATM complexity is not simply a matter of pumping 53 octet cells at OC-48 speeds, it is a non-stop juggling act in which network collapse can occur if the ATM switch can't keep up and starts dropping cells.

Because of these realities, ATM systems provide serious challenges for ATM testing platforms. A test system for ATM must be able to test the signalling functions, track multiple traffic flows over the same link, recognise the various encapsulation processes, capture non-frame traffic (eg, voice), oversee the policing and management activities of the switch and accommodate the need for multiple points of attachment. And all of this must be done at wire speeds, because it is at those speeds that the action takes place. Sampling at lower speeds misses the point and fails to adequately evaluate the network's performance.

Furthermore, because the different traffic sources include significantly different protocol demands, it is difficult to equitably match 'apples and oranges' on the same service. The trick of supporting inputs in a compatible manner with its service demands is daunting. And to test that ability is equally daunting. Test systems must have the capability to conduct automated analysis of protocol translation/mapping across multiple topologies.

Summary

ATM-based networks will become commonplace over the next decade both in the carrier's integrated multiservice networks and as part of user-owned and operated private facilities. ATM will be used to optimise the carriage of traditional network services (ethernet, IP, etc) and to serve as the basis for new high performance, high-speed services for the multimedia traffic of the future. Providing the assurance that an ATM network has been designed correctly, is operating properly and can continue to meet the challenges of a demanding traffic mix is only possible when sophisticated testing facilities are available. Testing is recognised as being an essential part of any successful ATM-based infrastructure and must not be overlooked when deploying a network.

Significant challenges exist for manufacturers of ATM equipment as well as for the operators of public and private ATM-based networks. These challenges must be met successfully if the benefits of deploying ATM are to be fully realised. Testing contributes to the solution in various ways including automating design verification, determining what levels of quality are actually being delivered, locating errors or outages, and providing analytical and historical support.

ATM networks that integrate multimedia traffic and aggregate legacy network services into a common infrastructure are the vision, but the reality is that no network is perfect.

OAM management capabilities and out-of-service system testing provide a powerful combination of tools to help the network manager. Automation of testing combined with standardised test cases, test scripts, and access to testing points provide a powerful toolkit for the test engineer. Rewards that go along with assuring high quality testing of ATM networks include:

• Flexibility to mix different traffic types, to make use of a wide range of physical media, and to select the level of quality necessary for the application (not just what the provider can offer);
• Scaleability to use a wide range of speeds (from 1.5 Mbps to 622 Mbps and above) and to expand in size from the desktop to the enterprise;
• Interoperability through the use of standards that are accepted by virtually all vendors and network providers, which allows multivendor networks, interconnection of different networks, and the integration of functions such as network management;
• Manageability enhanced with network integration as a result of fewer components to be managed, more consistent treatment of traffic, and the adoption of common standards; and,
• Economies of scale generated whenever more traffic can be combined onto a single set of network facilities, when the number of components to be managed decreases, and when fewer changes are needed during a network's life cycle.

• Share this Article:
• digg this
• add to del.icio.us
• StumbleUpon
• RSS Feeds