Monday, December 08, 2008
When you can't trust tcpdump
I just spent a good chunk of today bothering the telecom people to try and figure out why one server of mine couldn't talk to any off campus NTP servers. I had two servers, one was talking just fine, the other wasn't. For proof, I had packet traces showing that the non-working server was not getting an appropriate "ntp server" return packet.
And yet, after the telecom people sniffed the border firewall connections they saw UDP/123 packets on both sides. In other words, it transited the firewall just peachy. And also our IDS/IPS. So, clearly it was getting in. But it wasn't showing up on the tcpdump output on the server.
Then about 15 minutes ago I turned off the "restrict default ignore" line on that server.
And it started syncing off campus just fine. With packets.
WHY was tcpdump not showing the packets? That's what I want to know! Somehow, the UDP packets were being dropped before tcpdump saw them. Strange.
And yet, after the telecom people sniffed the border firewall connections they saw UDP/123 packets on both sides. In other words, it transited the firewall just peachy. And also our IDS/IPS. So, clearly it was getting in. But it wasn't showing up on the tcpdump output on the server.
Then about 15 minutes ago I turned off the "restrict default ignore" line on that server.
And it started syncing off campus just fine. With packets.
WHY was tcpdump not showing the packets? That's what I want to know! Somehow, the UDP packets were being dropped before tcpdump saw them. Strange.
