At the moment, our telephony people are having some issues with various things like handsets logging themselves out under certain conditions.
Vendor support has requested packet captures, and rather than schlepping around to the four corners of campus, dropping off laptops by each switch stack, I thought "wait, Junos does port mirroring, and it looks straightforward".
It is, and it isn't...
It is, and it isn't...
The documentation suggests it is as simple as
Simple.
Did that.
Got nothing.
Only saw the capture laptop's internet background radiation traffic.
I suspect this was because the destination port wasn't configured in ways that were helpful, either because the configured port type (perhaps family ethernet-switching was needed) or the interface-mode (configured as interface-mode trunk) not being set meant that Junos was not spitting out packets there. It should be noted that the input source interfaces I was playing with are tagged trunks with multiple VLANs on them, which may play a role; it may be that non-tagged interfaces behave differently (this suggests some lab experiments for another day).
So, once I set the capture interface to trunk mode, verily did my laptop drink from the firehose of the uplink ports of two large switch stacks during normal office hours:
You should usually also set your capture interface to grab egress frames (most people care about bidirectional traffic by the time they pull out wireshark). So for a complete capture solution for two interfaces, capturing ge-0/0/2 and ge-1/0/2 and outputting that to ge-0/0/14 with an analyzer called telephony-analyzer:
Obviously, you need to consider if the combined bandwidth will exceed the size of your monitor port and size correctly (or apply filters). At high enough packet rates, you may even need to think carefully about capture hardware. Fortunately, our packet capture requiring experiment is planned after hours during a maintenance window, so hopefully things will be calmer.
Another way of getting to the traffic you care about is only mirroring a VLAN - often the problem is restricted to a particular VLAN, and you'd like to dig into that, so instead of setting the ingress and egress interfaces, set ingress and egress VLAN ID, and the capture interface:
Another option you can use is to set up a port mirroring VLAN (aka remote monitoring), and there are ways of further filtering the captured traffic with firewall filters if you need to do that (see the further reading links below), but I've not explored either of these as I had easy physical access to a port, and the vendor requested no filtering of capture files (!?). Throwing mirrored frames into a VLAN is a pretty handy way of dropping traffic off where you'd actually like to receive it, particularly on relatively flat networks.
Despite the vendor saying "no, no, no, don't filter anything, just break the packet capture up in to 500Mb chunks (!)" I think one ought to carefully consider privacy and information security when supplying packet captures to 3rd parties (or even having them floating around on your disk for any length of time).
Sure, lots of things are https:// these days - but not everything in a LAN is, and some (V)LANs are sensitive (this one potentially contains much finance and HR traffic, and any ongoing voice calls). Post capture, you should therefore filter down to only that which is relevant - things like OUI filtering will be rather helpful for vendor specific troubleshooting - and only send them that.
Sometimes you need to see a packet capture before you know what is relevant, and if the vendor asks for more, you can filter it out and send it. Of course, normally you'd filter during capture (depending on how well you understand the likely problem), but sometimes following the letter of the vendor's instruction, and subsequently modifying it (with the option of going back to a "full" capture to tease out anything you inadvertently missed) is easier in the long run.
When you're capturing packets in Windows, it (in)conveniently strips VLAN tags (at least it did on the 3 different laptops I tested this on), so be sure you know what traffic belongs in which VLAN if you're capturing traffic from more than one at a time (IP addresses help, but when you're diagnosing L2 stuff like DHCP, it's not going to be too obvious).
When you're doing really big captures, you'll probably want to stream to disk using capture options, likely with the file size splitting option. Here, capturing a single VLAN at a time, or moving to a capture platform that preserves VLAN tags, would be useful. If you need to capture several simultaneously and not get them jumbled up, you should be able to set up (provided you have enough interfaces available) multiple port mirrors, one per VLAN. Alternatively, configure a device that doesn't discard VLAN tags to capture your frames (most linux-y things tend to be better at this, and tcpdump has plenty of useful options).
set forwarding-options analyzer your-portmirror-name input ingress interface <interface you want to monitor>
set forwarding-options analyser your-portmirror-name output interface <interface you'll plug your packet capture device into>
Simple.
Did that.
Got nothing.
Only saw the capture laptop's internet background radiation traffic.
I suspect this was because the destination port wasn't configured in ways that were helpful, either because the configured port type (perhaps family ethernet-switching was needed) or the interface-mode (configured as interface-mode trunk) not being set meant that Junos was not spitting out packets there. It should be noted that the input source interfaces I was playing with are tagged trunks with multiple VLANs on them, which may play a role; it may be that non-tagged interfaces behave differently (this suggests some lab experiments for another day).
So, once I set the capture interface to trunk mode, verily did my laptop drink from the firehose of the uplink ports of two large switch stacks during normal office hours:
set interface <your capture device interface> unit 0 family ethernet-switching interface-mode trunk
You should usually also set your capture interface to grab egress frames (most people care about bidirectional traffic by the time they pull out wireshark). So for a complete capture solution for two interfaces, capturing ge-0/0/2 and ge-1/0/2 and outputting that to ge-0/0/14 with an analyzer called telephony-analyzer:
set forwarding-options analyzer telephony-analyzer input ingress interface ge-0/0/2
set forwarding-options analyzer telephony-analyzer input ingress interface ge-1/0/2
set forwarding-options analyzer telephony-analyzer input egress interface ge-0/0/2
set forwarding-options analyzer telephony-analyzer input egress interface ge-1/0/2
set forwarding-options analyzer telephony-analyzer output interface ge-0/0/14
set interface ge-0/0/14 unit 0 family ethernet-switching interface-mode trunk(this assumes ELS platform syntax on EX gear, others may be subtly different)
Obviously, you need to consider if the combined bandwidth will exceed the size of your monitor port and size correctly (or apply filters). At high enough packet rates, you may even need to think carefully about capture hardware. Fortunately, our packet capture requiring experiment is planned after hours during a maintenance window, so hopefully things will be calmer.
Another way of getting to the traffic you care about is only mirroring a VLAN - often the problem is restricted to a particular VLAN, and you'd like to dig into that, so instead of setting the ingress and egress interfaces, set ingress and egress VLAN ID, and the capture interface:
set forwarding-options analyzer telephony-analyzer input ingress vlan <ID>
set forwarding-options analyzer telephony-analyzer input egress vlan <ID>
set interface ge-0/0/14 unit 0 family ethernet-switching interface-mode trunk
set forwarding-options analyzer telephony-analyzer output interface ge-0/0/14
Filtering is the better part of valour
Despite the vendor saying "no, no, no, don't filter anything, just break the packet capture up in to 500Mb chunks (!)" I think one ought to carefully consider privacy and information security when supplying packet captures to 3rd parties (or even having them floating around on your disk for any length of time).
Sure, lots of things are https:// these days - but not everything in a LAN is, and some (V)LANs are sensitive (this one potentially contains much finance and HR traffic, and any ongoing voice calls). Post capture, you should therefore filter down to only that which is relevant - things like OUI filtering will be rather helpful for vendor specific troubleshooting - and only send them that.
Sometimes you need to see a packet capture before you know what is relevant, and if the vendor asks for more, you can filter it out and send it. Of course, normally you'd filter during capture (depending on how well you understand the likely problem), but sometimes following the letter of the vendor's instruction, and subsequently modifying it (with the option of going back to a "full" capture to tease out anything you inadvertently missed) is easier in the long run.
When you're capturing packets in Windows, it (in)conveniently strips VLAN tags (at least it did on the 3 different laptops I tested this on), so be sure you know what traffic belongs in which VLAN if you're capturing traffic from more than one at a time (IP addresses help, but when you're diagnosing L2 stuff like DHCP, it's not going to be too obvious).
When you're doing really big captures, you'll probably want to stream to disk using capture options, likely with the file size splitting option. Here, capturing a single VLAN at a time, or moving to a capture platform that preserves VLAN tags, would be useful. If you need to capture several simultaneously and not get them jumbled up, you should be able to set up (provided you have enough interfaces available) multiple port mirrors, one per VLAN. Alternatively, configure a device that doesn't discard VLAN tags to capture your frames (most linux-y things tend to be better at this, and tcpdump has plenty of useful options).
Further reading:
- Configuring Mirroring on EX4300 Switches to Analyze Traffic (CLI Procedure)
- Example: Configuring Mirroring for Remote Monitoring of Employee Resource Use on EX4300 Switches
- vlan (Port Mirroring)
No comments:
Post a Comment