Tales from the crypt, a second look into the new SUNET.

av Fredrik "Hugge" Korsbäck den 23 Nov 2015

 

When we last took a peak into our lab we primarily  looked at the routers. But the routers is not everything. Since the new SUNET will be built upon national network of dark-fibre we need something that can transport light long distances. The tender for optical DWDM-equipment is fresh off the presses since now that the proposed equipment has cleared the initial test on both lab-fiber and on real fibre (Stockholm -> Västerås). The optical platform we will build the new network on is a gridless system from ADVA called FSP3000. The first stage of testing is in progress now in November and December and we wanted to share our initial test-protocol with the readers of this blog to provide a deeper insight in how one can do acceptance-testing of a new platform.

IMG_2058
Optical lab

EDIT: It was rumoured that someone got a seizure of the cablingmess. I can assure you it will not look at this in the real network, this was pure lab and the node is already decommisioned and sent back

——————————————

This POC (Proof of Concept) is to verify that it’s possible to run Junipers Coherent 100G DWDM interface on Adva DWDM system with the optical design that is set for the new SUNET network.

One important part is to be able to share a dark fiber pair between the central PoP and the University handoff. The services running over the dark fiber are a 100GE grey LR4 wave, one 100G Coherent and one 100G supplier 10x10GE muxponder. The hardware at the university needs to be able to handle 4-5 Coherent waves from day one. The attenuation of the dark fiber is between 1 to 9dB and one case of 15dB. The 10×10 Muxponder will only be used on three different places in Sweden when the network is finished.

Test setup

Router

The Juniper hardware will be two Juniper MX routers with the following setup in each device.

  1. 1x 100G Coherent interface
  2. 1x 100GE LR4 interface
  3. 1x 10x10GE LR interface
  4. 1x 100GE ER4 interface

Tester

Tester, Two EXFO FTB-500 with 100GE IQS-85100G and 1x10GE interface will be used to verify bit errors and packet loss. Provided by SUNET.

IMG_2065
100km fibre in spools

DWDM

The DWDM node should simulate

  1. Two ”högskolenod” (Customer side), with accessfiber.
  2. Two ”stamnätsnod” (Core Pop)
  3. One ”förstärkarnod” (ILA)

1st_adva_dwdm

Test procedures

All optical connecters should be cleaned and inspected with fiberscope. The optical level should be measured on the ingress to all equipment and be compared to the optical levels read from the equipment management system and documented in the POC excel document.

Router setup verification

Before connecting any DWDM equipment the router interfaces should be tested. All three types of interfaces should be verified. Test should be run for 10x10GE, 1x100GE LR4 and 1x100G Coherent and also traffic over all the interfaces at the same time. Traffic could be run over ip or ip/l2 MPLS LSPs. Tests should be run for 1 hour for each individual test and 8 hours for the combined test.

 

Test Type Comment Verified
T2 100GE LR4 PASS
T3 100G Coherent PASS
T4 10x10GE PASS
T5 Load balanced traffic over all links PASS

2nd_adva_dwdm

 

DWDM verification

This test shall verify that Juniper Coherent interface can run as an alien wave on the DWDM system. That coherent waves can share the fiber between the Customer-site and the POP with a 100GE LR4 or ER4 interface. In the case that 100GE LR4 does not work with 7dB attenuation, verify the highest attenuation that LR4 works with and the swap the LR4 interface with ER4 and redo the test.

Waves

The 10x10GE muxponder 100g-wave and the Coherent 100g interface should be using the channels next to each other with the longest possible wavelength.

3d_adva_dwdm

 

The same test as without DWDM equipment shall be run.

 

Test Type Comment Verified
T6 100GE LR4 PASS
T7 100G Coherent PASS
T8 10x10GE PASS
T9 Load balanced traffic over all links PASS

 

All available Error counters shall checked and be documented, both the type of counter and the value. A special pre FEC BER values should be checked.

Access fiber breaking point

This test is to verify the headroom that is available on the access fiber. The attenuation of access fiber A should be increased until bit errors occurs on the Tester with traffic flowing over all three link types. The attenuation value should be noted and verified which link type that fails first. Increase the attenuation until all three types has bit errors.

Wave collision

This test is to see how the DWDM and router system handles two coherent waves running on the same channel. Set the coherent 100G  interface to the same channel as the 10x10GE Muxponder.

Test Type Comment Verified
T10 Coherent and transponder on the same frequency PASS

Gray 1550nm on add drop port

This test is to verify what will happens to the coherent waves when by mistake a gray 10GE 1550nm interface is plugged in to the add/drop port at the university. SUNET will provide 10GE SFP+ on 1550nm.

Test Type Comment Verified
T11 Coherent colliding with gray 1550nm. PASS

Fiber abuse

This test is to verify how much abuse the fiber can withstand without generate bit errors. This test is very un scientific but is just to get some feel on what to expect. Set the attenuation of the access fiber A to 1dB below the value found in previous test when BER occurred for the Coherent interface. Bend, smash and abuse the fiber and document the result with video and interface counters. CLICK HERE FOR VIDEOS 

Test Type Comment Verified
T12 Fiber abuse PASS? 🙂

Short distance 10GE alienwave

This test is to verify if it’s possible to run a 10GE DWDM alienwave over a non DCM compensated network over a short distance parallel with the coherent waves. The 10GE wave will be produced using Menara OTN XFPs. SUNET will supply the optics and the Juniper MICs and the passive DWDM filters. The DWDM filters should be connected an add/drop port in the Roadm. The 10GE wave will be used in two locations to be able to have two independent aggregation switches at one PoP. The maximum distance between the pops in the network are 92km, without and ILA in between.

4th_adva_dwdm_alien

Field test

This test is to verify that the coherent interfaces works on the aerial fibers that SUNET are going to use. One of the Pop nodes is going to be moved 100km from the site to a real PoP(Västerås) and the ILA are also going to be moved to a real ILA site(Hamra). The rest of the equipment is remaining at the test location.

5th_adva_dwdm

Test should be run for at lest 24hours, Performance measurement values should be polled every 2-5min.

See more pictures from the field-test

Summary

All in all the tests went accordingly to plan. All equipment kept to their documented specifications and during weird scenarios the equipment behaved as expected. We also managed to run the Coherent wave together with a 10×10 Muxponder Coherent 100G wave AND the 10G alienwave without using any guard-bands in between the channels. This is not reccomended by the vendor but its interesting to see that it works from a technical point of view.

This pretty much sums it up for physical tests before christmas. We still have tons and tons of acceptance-testing to do and we have not touched on the software-part yet. There is a lot of integration and interop regarding software between vendors and our existing tools to be done. This will be picked up immediately after new-years where we will build up the lab again but then focus on logical networking instead of physical networking .

Live long and prosper

1968-01-01

Fiberfeber startades

1 May 2015

Beställa routrar och optik

15 Sep 2015

Leverans av routrar och optik

1 Jan 2016

Byggnation av kärnnät

1 Feb 2016

Börja flytta över kunder

16 Jun 2016

Nya Sunet operativt

16 Oct 2016
2019-10-18

Fredrik "Hugge" Korsbäck

Network architect and chaosmonkey for AS1653 and AS2603. Fluent in BGP hugge@nordu.net


Warning: file_get_contents(https://www.linkedin.com/countserv/count/share?url=https://www.sunet.se/blogg/tales-from-the-crypt-a-second-look-into-the-new-sunet/&format=json): failed to open stream: HTTP request failed! HTTP/1.0 404 Not Found in /var/www/html/wp-content/themes/sunet/functions.php on line 5416