Inside Interop Blog

Thanks to everyone in the NOCs and in the Interop Labs for contributing to such a fantastic event last week! I hope you’re all resting up after so many long days. Hope the tear-down went smoothly over the weekend.

The final batch of interviews from the show last week are up, and can be found here:

http://interop.libsyn.com/

The show floor closed yesterday and after about an hour of tear down, our job was done. Of course, we kept on working till the very end, spending most of yesterday backing up config files and making little notes for documentation. Timmons was finally allowed to try and break our routers, which gave us some very interesting test results. This has been a great show and I think everybody on the team is going home with a lot of new knowledge and hoping we can do this again next year since there is tons of interest and so many things going on in the open source are.

Thanks to all who visited us in the booth, hopefully we will see you again next year!

It is with the normal bittersweet sense of another show behind us that the InteropLabs teams wrapped up the day today. Since I am flying to another seminar this weekend, I’m sitting at McCarron airport typing this to you after the final day of the show and teardown. We had great response from the folks who dropped by the InteropLabs area on the show floor (although none of you admitted to reading the blog!), and the interest in NAC is very strong and growing.

I am deeply appreciative to the NAC team for allowing me to take on the leadership role this year, and they’ve been incredible: Karen O’Donoghue (our Educator, whose name I managed to misspell on the signs, even though I’ve known her for about 10 years, and who has led the previous incarnations of this lab), Joel Snyder (Network World writer, consultant, and a stickler for details), Jan Trumbo (consultant and the person responsible for the great graphics we had), Craig Watkins (long-time NOC Team member, consultant, and a guy willing to take on whatever we needed), Kevin Koster (NAC developer and the guy who took on the creation of the great Flash presentation in the area as well as diving into NAP), Brett “Thor” Thorson (IPv6 jock and the guy who took on the Cisco NAC work), Chris Hessing and Mike McCauley (open/full source authors who took on creating a TNC-compliant open source project for the NAC area of the InteropLabs), and the great engineers from the various contributing companies who joined us and took on all of the weird requests that we made of them.

Thank you all…

At this point, we have torn down all of the gear, boxed it up, placed it into the custom-designed “D-crates”, and left them to be shipped back to the Interop warehouse in California. At this point, our plan is to set the NAC Lab up again sometime in late July or early August in hot staging for Interop New York in September.

Will you be there? If you didn’t make it to Las Vegas, you really should plan to join us in NY.

Who do you let onto your network? How do you know that they are acceptable? How do you know you’re keeping the “bad guys” off your network? All of these questions and more will be the focus of the NAC education that you’ll see at Interop New York.

Until then… Cheers!

It is with great honor that we wrap up this series of Blogs with an interview with Neal Allen, TAC Level 3 Escalation Engineer at Fluke Networks, a long time veteran of InteropNet, both as a member of the NOC team and also a corporate sponsor.

Q: Neal, in this series of Blog, I try to trace back the steps of InteropNet and SpyNet and also the people who worked hard to make this event a reality. From the horse’s mouth, why did we need SpyNet to begin with? What was InteropNet like 10 years ago? I am going to truncate it to 10 years, if you don’t mind. I know you go back further than that.

A: In a twisted sense, the “old network” was much more fun. Yes, we were sometimes in the convention center for upwards of 20 hours per day for many days before the show opened, but it was so cutting edge we are still bleeding years later. There were times that the compile time stamp (not date) in the router code was some fifteen minutes before the show opened. It was an exciting time. The name of the show “Interop” was selected to portray the interoperability possible with the then generation of technology, which often didn’t play well with others. To accomplish this we usually had the development engineers from the various router and switch vendors sitting side by side with development platforms compiling new code all week to force the switches and routers to work together. Imagine the development engineers from 3COM, Wellfleet, Proteon, etc., all elbow-to-elbow trying to discover why the frame didn’t make it. This show single-handedly contributed significantly to the possibility for multi-vendor networks to operate.

But back to the Spy Network …

Because the show is distributed fairly widely over the convention center, and because we don’t have the luxury of telling the user to “just wait a few minutes”, we instituted the Spy Network to permit the NOC staff to have a media-level link to any part of the network. This was deployed using a parallel cable plant to avoid single-points-of-failure. In Telco terms, we didn’t have to roll a truck. This permitted us to see MAC layer errors and traffic from anywhere without leaving our comfy chairs in the NOC. At the time this was deployed it was pretty revolutionary. These days people set up management VLANs to accomplish similar things, but that does not provide the same level of access, and it goes through the same trunk links as the user traffic.

Of course, due to the nature of the network, there were as many as sixteen parallel networks to deal with, so Spy was in constant use and the biggest problem was determining which protocol was coming out of the link so that you connected the right analysis tool. We used the latest bleeding edge technology as the primary network transport, but always had one or two tried-and-true backup technologies just in case.

Q: That’s great stuff. So over the last ten years, what happened to networking and what happened to network troubleshooting and monitoring?

A: When I first started with the Interop network it could be described as large broadcast domains separated by a few routed connections. That made it very easy to diagnose with a traditional protocol analyzer. Today’s network is entirely switched, and is exceedingly difficult to monitor and troubleshoot. It isn’t good enough to monitor an uplink any longer. As with a typical corporate network we are now faced with having to determine where to start. Only we have a few Alpha or Beta products in exhibitor booths which may or may not be working correctly at this first unveiling, which the normal corporate network usually does not experience.

* Is it the client data?
* Is it routing?
* Is it a loop or alternate path between a VPN to the corporate network and the normal path out of our network?
* Is it traffic from another exhibit?
* Is it some form of virus or Internet based attack?
* Is it the network protection (authentication or firewalls) within our network?
* Is it marginal or faulty cable? Or over saturated wireless…
* Is it a marginal or failed hardware element in our network?
* Is it a Beta product which does not yet operate correctly, but is being introduced at the show?

and so on.

The current trends toward converged networks, and toward having more intelligent switching products, I am seeing a huge need for the diagnostic tool vendors to be even more creative than the product vendors in inventing a few new methods for monitoring the network and for unearthing increasingly more exotic problems. I am not sure what direction that will take, but the need is evident.

Q: Now the big question. What’s next for Spynet and monitoring? What do you see as the big breakthrough going forward?

A: This show has recently been using tapping technologies which are as complex as the switch/routers being monitored. That is a large step in the right direction, but it isn’t enough. I think we will have to get “inside” the switch/router itself and interact with the configured intelligence of the box itself in order to make the next leap forward in monitoring. However, we will still need to see the resulting data stream in and out to both double-check on the box, and to offer adequate monitoring, alerting, and protection.

The Spy Network is needed as much as ever before, perhaps more so. Otherwise how will you know when the switch itself becomes confused? But it will be partnering with “inside the box” monitoring and diagnostics sometime soon.

Thanks, Neal. You have been an inspiration for all of us. And I am sure I will see you again in New York.

Denny K Miu
Gigamon Systems

Part 1: InteropNet - Tribal Customs and Best Practices
Part 2: History of SpyNet (Son of LAN-Hopper)
Part 3: Interop*Spy*Net
Part 4: SpyNet and Network Physics
Part 5: SpyNet and Internap
Part 6: SpyNet and Neal Allen

This following is an interview with Eric Klinker, the CTO of Internap which is a sponsor of InteropNet (specifically, SpyNet).

Q: Eric, Internap has the distinction of being the provider of the first and only 10G monitoring tool to SpyNet. What does the Internap box do?

A: The Internap FCP 10G is a new 10Gbps version of the award winning Flow Control Platform, and is designed to deliver the highest level of performance and risk mitigation for the largest of networks. As you know, InternopNet is connected to the Internet with two redundant high speed links from two different providers and the FCP’s primary function is to balance between the two. By leveraging the best aspects of each of Interop’s providers, the FCP 10G is able to deliver a much better network experience for the show’s participants and test labs. To avoid unacceptable and costly downtime, the FCP 10G’s intelligent algorithms ensure that Interop’s network is available even when one provider is completely down or performing poorly.

Q: Why is 10G important for this technology?

A: The FCP 10G is designed for an emerging and demanding class of customers that have rapidly growing applications and networks. Large e-commerce operations, social networking sites, and emerging media distribution networks are among the first to demand this level of performance from the FCP. We clearly expect more to follow as 10G gains wider acceptance in the switching fabrics of the future.

Q: Why would the attendees of the show be interested in this technology?

Interop has long been the showcase event for emerging network technologies. If an enterprise relies upon the Internet for core aspects of their business, they come to Interop each year in search of new ways to better run those businesses. If the network is mission critical to an enterprise, it’s important to do two things: leverage more than one network provider through multi-homing, and maximize the performance of those networks with intelligent routing. Interop is the perfect venue to demonstrate these principles and the FCP remains the intelligent routing platform of choice for mission critical applications.

Thanks, Eric, that helps a lot and hope to see you again in New York.

Denny K Miu
Gigamon Systems

Part 1: InteropNet - Tribal Customs and Best Practices
Part 2: History of SpyNet (Son of LAN-Hopper)
Part 3: Interop*Spy*Net
Part 4: SpyNet and Network Physics
Part 5: SpyNet and Internap
Part 6: SpyNet and Neal Allen

I can not believe it is already the end of the second day and only one more day to go. These two days have really passed quickly and yes, we are most definitely having fun. Despite the fact that the ilabs booth is in the corner we seem to have a pretty constant, good flow in the oss area. It feels like I’ve been talking constantly for two days, which is probably pretty accurate, and so have most of our team members. And everybody I have talked to are really interested in what we are doing, asking lots of questions and spending a considerable amount of time here. It is really a lot of fun when people are so positive about what we’re doing. If anybody was every in doubt that open source was a hot topic, don’t be.

As is always the case at the end of the second day, we wish we had a few more days to work on this. But, today is the last day, and tomorrow at 3pm we’ll tear everything down. Before that, however, we will do some serious testing. I have promised Timmons that he will be allowed to attemt killing quagga, xorp and bird and simulate some attacks on our network with all his toys. We might wanna wait untill after lunch, though, before we start breaking our network, hopefully we’ll get a good crowd here tomorrow as well.

In my final podcast interview at Interop Las Vegas ‘06, I chat with Tom Burns, who’s the senior vice president and general manager of Alcatel’s enterprise network infrastructure group. Last week Alcatel launched the OmniSwitch 6850 family of stackable 10/100/1000/PoE, 10-Gigabit workgroup switches, which are the first Alcatel switches to provide Gigabit power over Ethernet (PoE). Yep, the switch market is getting fairly commoditized, but Burns explains how Alcatel will to try stay ahead of the price-performance curve by adding new functionality like POE and security to its switches.

Here’s the podcast…

Read the rest of this entry »

One of the big fish at Interop, Nortel, is using the show talk about the company’s new Secure Router Portfolio. While this family of routers was announced in late March, it’s a major focus for Nortel at Interop this week. The Secure Router family is a direct result of Nortel’s acquisition of Tasman Networks in December, and contains the 1001, 1002 and 1004 models for small offices and branch deployments, and the 3120 for mid-range deployments. In this interview, Aziz Khadbai (VP and GM of the Converged Data Networks Group) and Bob Reason (Security and Routing Marketing Manager) talk about the new line of routers.

Here’s the podcast…

Read the rest of this entry »

The following is an interview with one of the sponsors of the InteropNet, in particular, a sponsor of the SpyNet, John Harriman who is a Senior Director of Marketing at Network Physics.

Q: John, I understanding that this is the second year that Network Physics is sponsoring InteropNet or SpyNet, can you talk about what your product does and why it is important for InteropNet and how it is ultimately interesting for the attendees of Interop?

A: Network Physics provides application management for the network team. Our appliance discovers all applications running on the network, then identifies network issues as fast as the network engineer can move their mouse. Our primary product feature, NetSensory Insights (point-and-click action guides that encapsulate best practices for the particular monitoring task at hand), is designed to pinpoint the source of a problem and quickly troubleshoot response time issues. This is useful to the InteropNet NOC team to discover network issues before they effect the quality of the event. It is interesting to the attendees because it gives them a quick insight into Interop networking issues that could also show up on their own networks.

Q: So what kinds of traffic are you interested and how do you bring that information back to your booth so that it can be presented to a live audience?

A: The NetSensory appliance in the NOC captures all IP traffic, TCP and UDP, flowing through the network. Normally, our product sits passively on the network attached via SPAN or mirror port or tap (no agents, no SNMP, no polling, no synthetic transactions). In the case of the InteropNet, the Gigamon switch is performing aggregation, multicasting and load-sharing so that we can share the traffic with other sponsors. Once we capture the traffic, analysis is performed and resulting metric is accessed via a Java-based console on the network engineer’s laptop back at our booth.

Q: Is there any similarity between what goes on at the InteropNet and what goes on at your typical customers’ data centers?

A: Yes, Denny. Admittedly the InteropNet is a bit of an “anything goes” network and more ad hoc than a typical customer data center, but customer’s networks are not immune to the challenges that appear here at the show. Two examples come to mind:

Inappropriate use of Bit Torrent or similar P2P applications can consume unreasonable amounts of bandwidth, leading to the premature impression that the network requires an increase in bandwidth when all that’s really called for is an email to the manager of the vendor involved.

Another example involves worms. Last year at Interop our “Worm Hunt” Insight detected the Blaster worm. Just a few clicks identified the source: a single computer at one exhibitor’s booth - probably a road warrior bringing in an infected laptop. Because it detects behavior common to all worms, rather than pre-programmed signatures, the network engineer can use it to catch day-zero infections, see at a glance which computers are infected and which computers they’re trying to infect.

Thanks, John, and look forward to see you again in New York.

Denny K Miu
Gigamon Systems

Part 1: InteropNet - Tribal Customs and Best Practices
Part 2: History of SpyNet (Son of LAN-Hopper)
Part 3: Interop*Spy*Net
Part 4: SpyNet and Network Physics
Part 5: SpyNet and Internap
Part 6: SpyNet and Neal Allen

Here’s an interview I did today with Mahboud Zabetian, the President and CEO of WildPackets. This week at Interop, WildPackets is doing the first public demos of its Omni Distributed Network Analysis Platform 4, which has built-in analysis for VoIP, application monitoring, and network forensics. Zabetian explained to me how the product can help optimize network services and maximize the uptime for enterprise networks.

Here’s the podcast…

Read the rest of this entry »