This series isn't well-named, or even following good numbering standards, as it is a problem in motion. You know, the kind where you have a feeling of success when you don't solve it exactly, but you make it move around in time or space. The prior posts were called:
- Return of the ABAP Detective - A Case of Peripheral Damage
- Chapter Two: The ABAP Detective Vs. the Dragon Network
This post is chapter 3, merely because I've got 2 case folders open already, and this latest evidence won't fit as a comment to either of them, and though the case isn't solved yet, document more clues may break it wide open.
pictures of device 1
The above 2 eye charts show the results of one stakeout. I'm unable to keep going 24 hours a day, so I wire up bots to record data for me. It's like having a wiretap, or video surveillance, no warrant needed, just need to keep good notes to avoid fingering the wrong culprit, and either be a speed reader, or have a technique to fast forward to the incriminating part.
On the left side are measurements in milliseconds. On the X-axis are times in minutes and seconds. The data was gathered from traceroute commands, with the resulting log files rearranged to fit Excel text conversion easily. Okay, if not easily, then with a minimum of manual labor. Once in Excel, I cast the data columns into a high-low-average order that seemed to fit nicely into one of the preexisting chart types (Stock Market: High/Low/Close). All right, big shot, the police captain says, what's the point? I would says that I have eliminated a suspect, this device is clean, no evidence of wrong doing. However, we'll leave that for the grand jury to decide. All I know is, I didn't catch this pair in the act, while others were visibly corrupt. Let's see the next pair of mug shots, eh?
pictures of device 3
Hey, where's the accomplice? Oh, nothing to hold them on so we let them go. Well, what have we got? For 5 minutes, between 2:34 and 2:38 AM, elevated round trip times. This joker was in orbit. Whether it was pixie dust or some other diversion, we can't tell without an autopsy or a confession, and we don't seem to have a chance for either of those. At least we have times to look for chronological overlaps, right?
Before I forget, this is the old "3" not the new "3"; I'll fill in that tangent later.
Right, next group. Bring them in, don't slouch, take off your hat, step up to the line and state your name and IP address.
pictures of device 5
More dirty business, around the same time, though this pair of jokers had an even wilder time that the last one. From 2:30 until 2:40, and then 2:42 up through 2:48, these packets were sky high, at 250 milliseconds round trip. That will take you halfway around the world on a normal ride. What are these hoods up to?
All right, put them back on the streets, but keep an eye on them. Who do we have next? More usual suspects? Line them up.
pictures of device 9
Device number 9? Number 9? Number 9? What happened to 7? All right, no matter, what do these look like? Quite the busy pair, and I believe there's another like them as well that got away. One side of the couple was skating like the rest, while the other was dormant, probably playing lookout, the patsy. But I can still discern a bit of jumpiness there. Can't explain it, but it's around the same time. Must have been quite the jumbo packet party during that window of opportunity.
Last call; anyone else in the paddywagon? One more, sarge. Ah, sorry, this isn't quite a pair. One from each side of the street, if you will. Neither a cause for concern.
pictures of device 11
pictures of device 12
Informers, er, qualified sources
Not all suspect activity can be discovered by sleuthing, or poring through mug shots. At times, the detective must turn to sources, which might be known as insiders, or squealers. Forums can be a place to find them; as this is more of a hard case than a soft (ware) one, I would not expect to find useful answers in the SCN space. I'd look on the switch and router vendor space. The switch yards are a tough place, with jargon all their own, and as unwelcoming to nØØbs as a UNIX kernel channel on IRC.
I put together a wanted poster, disguised as a Google search, but ended up with thousands of hits on "pause input". After drinking slugs of coffee, I realized I needed to narrow my search not to just any pauses, but to those beyond 0.
- "pause input" -"0 -pause -input"
That line, plus the vendors name, got me 23 hits, a much more helpful set of reports to peruse than the entire encyclopedia of the internet. A specific forum link showed me symptoms so close to what we've been seeing it was almost like looking in a mirror. On the other hand, it didn't show the arrest and conviction records I was hoping for, just the same "most wanted" description I was working on.
Going to the internet public library, as it were, gave me more background, with an article on Ethernet Flow Control practically demanding to be read over and over, like a Raymond Chandler novel. And yet, there's that nagging feeling that the evidence might be pointing in one direction, but we don't have a confession, everything is circumstantial. One particular quote from a vendor troubleshooting guide says it this way:
| Although the values of the counters are largely accurate, they are not very precise by design. In order to pull the exact statistics of the traffic, it is suggested that you use a sniffer to monitor the necessary ingress and egress interfaces. |
Yikes! We're looking at numbers that might not be correct, and we'll need to send in an undercover agent (called a "sniffer" by this mob) to gather more dirt? I'll leave it here for now with this further quote from the same source:
| "Make congestion avoidance a priority on your network." |
1 Comment
