pantz.org banner
Configuration of NetFlow, Flowtools, pfflowd on OpenBSD
Posted on 05-16-2006 06:23:00 UTC | Updated on 04-05-2013 21:07:23 UTC
Section: /software/flowtools/ | Permanent Link

Have you ever wanted to keep track of every packet going through your firewall? How about getting some stats on the hosts using your network. Stats like most bandwidth used or most popular ports or ip's. Well NetFlow is what your looking for. NetFlow is an open but proprietary network protocol developed by Cisco Systems to run on Cisco IOS-enabled equipment for collecting IP traffic information.

Flow-tools is a suite of programs that takes Cisco NetFlow datagrams and generates reports based on the information it has recorded. The Cisco Netflow datagrams are generated by a program called Pfflowd.

The Pfflowd website describes it's programs functionality the best by saying: Converts OpenBSD PF status messages (sent via the pfsync interface) to Cisco NetFlow datagrams. These datagrams may be sent (via UDP) to a host of one's choice. Utilising the OpenBSD stateful packet filter infrastructure means that flow tracking is very fast and accurate.

Note: The PFflowd only accounts packets that get passed statefully. Since flow reporting is coupled to PF's state tracking, only traffic flows which are passed via a "keep state", "modulate state" or "synproxy state" rule are accounted. Blocked packets are not accounted for.

Below are some simple steps to getting this going. This install was done on OpenBSD 3.8.

1. Install python 2.4 from the version of OpenBSD your using. Follow the install instructions and make it the default Python for the system.

pkg_add -v http://openbsd.mirrors.pair.com/ftp/3.9/packages/i386/python-2.4.2p0.tgz

2. Install the Pfflowd package on any version of OpenBSD before 3.9.

pkg_add -v http://openbsd.mirrors.pair.com/ftp/3.8/packages/i386/pfflowd-0.6.tgz

If your using OpenBSD 3.9 then you will need to download the latest version from CVS from the authors website. A fix was committed to CVS on the authors site in June 2006 to fix it. If you need that the site is here. Just download the Makefile, pfflowd.8, pfflowd.c, and pfflowd.h into a directory. Then just issue the "make" command. It should compile no problem in OpenBSD 3.9. Then copy the made binary to /usr/local/sbin/ and the man page to /usr/local/man/man8/. Last step is to create the user "_pfflowd". If you don't then Pfflowd will not start.

3. Install the flow-tools .68 or higher. As of OpenBSD 3.8 & 3.9 you will need to download the source and compile it as both only have version .67. The reason for this is we will need the new flow-rptfmt program that only comes with .68 and later. You can get the source from the flow-tools site. If any later version of OpenBSD comes with .68 or later just install the package and forget the source install. The commands to make and install flow-tools are below. This will install flow-tools to /usr/local/netflow.

cd /tmp
wget ftp://ftp.eng.oar.net/pub/flow-tools/flow-tools-0.68.tar.gz
tar xvzf flow-tools-0.68.tar.gz
cd flow-tools-0.68
./configure
make
make install

4. Let's make the directory for the log files then start the flow-tools collector. This will listen on localhost for Netflow datagrams sent from Pfflowd. It uses the program flow-capture from the flow-tools package to collect datagrams sent from Pfflowd. It will keep binary logs of the collected datagrams in /var/log/netflow. It will begin a new file every 10 mins. It listens on localhost port 12345. Make sure you allow this in your PF rules if need be.

mkdir /var/log/netflow
/usr/local/netflow/bin/flow-capture -w /var/log/netflow -n 143 127.0.0.1/0/12345

5. Let's start Pfflowd so it will start sending data to the flow-capture daemon. This starts the Pfflowd daemon which will send Netflow datagrams to the flow-tools daemon on localhost port 12345. You might need bring up the pfsync0 interface so this will work. So we will do that also. To get it to come up on reboot just make the file /etc/hostname.pfsync0 and put the work "up" on the first line. Then save it.

ifconfig pfsync0 up
/usr/local/sbin/pfflowd -n 127.0.0.1:12345

6. Now that it's collecting data we will need to create a config file that will filter the data before we generate a report. The following lines need to be put in a file to be accessed by reports.conf config file. The name of my file is nfilter.conf. This is an example of a filter that could be used to narrow down the logs to ip ranges and ports if you would like to.

############################
filter-primitive popularports
 type ip-port
  permit 80
  permit 443
  permit 20
  permit 21
  permit 22
  permit 25
  permit 53
  permit 110
  permit 143
  default deny

filter-primitive intif
 type ip-address-prefix
  permit 192.168.0.0/24
 default deny

filter-primitive intif-ip
 type ip-address
  permit 192.168.0.10
  default deny

filter-primitive 10min-ago
 type time-date
 permit lt 600 seconds ago

##############################
filter-definition intif-src
 match ip-source-address intif

filter-definition intif-dst
 match ip-destination-address intif

filter-definition intif-src-popular
 match ip-source-address intif
 match ip-destination-port popularports

filter-definition intif-ip1
 match ip-source-address  intif-ip

7. Now that we have filters for the data we need to generate the reports from the filtered data. The following lines need to be put in a file and called from the command line. I named mine reports.conf.

#The filter lines below refer to this file
include-filter /etc/nfilter.conf

#### Start: Source IP -> Dest Port. Filter Name: intif-src ####
stat-report srcip-dstprt
 type ip-source-address/ip-destination-port
  filter intif-src
 output
  sort +octets
  records 15
stat-definition srcip-dstprt
 report srcip-dstprt
#### End: Source IP -> Dest Port. Filter Name: intif-src ####

#### Start: Destination IP -> Dest Port. Filter Name: intif-dst ####
stat-report dstip-dstprt
 type ip-destination-address/ip-destination-port
  filter intif-dst
 output
  sort +octets
  records 15
stat-definition dstip-dstprt
 report dstip-dstprt
#### End: Destination IP -> Dest Port. Filter Name: intif-dst ####
#### Start: Source IP -> Dest Port. Filter Name: intif-src-popular ####
stat-report srcip-dstprt-pop
 type ip-source-address/ip-destination-port
  filter intif-src-popular
 output
  sort +octets
  records 15
stat-definition srcip-dstprt-pop
 report srcip-dstprt-pop
#### End: Source IP -> Dest Port. Filter Name: intif-src-popular ####

8. Now we need a script to tie all this together so it runs the programs from flow-tools to generate the reports in html format. This is a korn shell script. Change the variables on the top to fit your needs. The output is very simple and can be changed to suit your needs. Save this to a file and execute it (chmod +x filename if need be). This should dump a very simple html file to the path you set in the script. It will show the output from the current day it is run.

#!/bin/sh

# Add to your path statement in /root/.profile: /usr/local/netflow/bin/

# Variables for settings below
REPORTPATH=/var/www/htdocs
CFGFILES=/etc
LOGPATH=/var/log/netflow
HTMLFILENAME=netflow.shtml
YEAR=`date "+%Y"`
MONTH=`date "+%m"`
DAY=`date "+%d"`

createreport () {
 flow-cat $LOGPATH/$YEAR/$YEAR-$MONTH/$YEAR-$MONTH-$DAY/ \
 |flow-report -s$CFGFILES/reports.conf  -S $1 \
 |flow-rptfmt -fhtml >>$REPORTPATH/$HTMLFILENAME
 echo "<br><br><br>" >>$REPORTPATH/$HTMLFILENAME
}

# Create html file
echo "<html><center>">$REPORTPATH/$HTMLFILENAME

# Run Create file function with report name
createreport dstip-dstprt
createreport srcip-dstprt
createreport srcip-dstprt-pop

# Close html file
echo "</center></html>">>$REPORTPATH/$HTMLFILENAME

9. Once you have verfied the script works then put it in a cron job to run every night. Since the script looks at the current day it is run I like to run mine right before the end of the day. Use the command "crontab -e" and insert the line below. This runs the script at 11:51 PM every night.

51      23      *       *       *       /var/www/cgi-bin/reports.sh >>/dev/null 2>&1

View the file netflow.shtml your favorite web browser to see the stats.

That is a simple example. But I wanted an email with the stats every night. I did not find a way to get it into a nice format so I wrote a perl script to put the output in a nice e-mail friendly format. The script takes the output from flow-report and gives the "octets" column more human readable numbers in Bytes, Megabytes, and Gigabytes. It also lines up all the columns neatly and just spits out acsii. You can then redirect the output of the script to an email that can be sent each day. The script will read all of the current days (the day it's run) stats. It's rough but it gets the job done. Just run the script by itself to see the output on the command line. Don't forget to put in the report file below in /etc/

Below is the report file I use. I use the file name and path /etc/report.cfg. There is no filter being used.

stat-report ip-address
 type ip-address
 output
  sort +octets
  records 15
stat-definition ip-address
 report ip-address

stat-report ip-source-address
 type ip-source-address
 output
  sort +octets
  records 15
stat-definition ip-source-address
 report ip-source-address

stat-report ip-destination-address
 type ip-destination-address
 output
  sort +octets
  records 15
stat-definition ip-destination-address
 report ip-destination-address

stat-report ip-source/destination-address
 type ip-source/destination-address
 output
  sort +octets
  records 15
stat-definition ip-source/destination-address
 report ip-source/destination-address

stat-report ip-source-port
 type ip-source-port
 output
  sort +octets
  records 15
stat-definition ip-source-port
 report ip-source-port

stat-report ip-destination-port
 type ip-destination-port
 output
  sort +octets
  records 15
stat-definition ip-destination-port
 report ip-destination-port

stat-report ip-port
 type ip-port
 output
  sort +octets
  records 15
stat-definition ip-port
 report ip-port

stat-report ip-source/destination-port
 type ip-source/destination-port
 output
  sort +octets
  records 15
stat-definition ip-source/destination-port
 report ip-source/destination-port

#########

stat-report ip-source-address/ip-source-port
 type ip-source-address/ip-source-port
 output
  sort +octets
  records 15
stat-definition ip-source-address/ip-source-port
 report ip-source-address/ip-source-port

stat-report ip-source-address/ip-destination-port
 type ip-source-address/ip-destination-port
 output
  sort +octets
  records 15
stat-definition ip-source-address/ip-destination-port
 report ip-source-address/ip-destination-port

stat-report ip-destination-address/ip-source-port
 type ip-destination-address/ip-source-port
 output
  sort +octets
  records 15
stat-definition ip-destination-address/ip-source-port
 report ip-destination-address/ip-source-port

stat-report ip-destination-address/ip-destination-port
 type ip-destination-address/ip-destination-port
 output
  sort +octets
  records 15
stat-definition ip-destination-address/ip-destination-port
 report ip-destination-address/ip-destination-port

stat-report ip-source-address/ip-source/destination-port
 type ip-source-address/ip-source/destination-port
 output
  sort +octets
  records 15
stat-definition ip-source-address/ip-source/destination-port
 report ip-source-address/ip-source/destination-port

stat-report ip-destination-address/ip-source/destination-port
 type ip-destination-address/ip-source/destination-port
 output
  sort +octets
  records 15
stat-definition ip-destination-address/ip-source/destination-port
 report ip-destination-address/ip-source/destination-port

stat-report ip-source/destination-address/ip-source-port
 type ip-source/destination-address/ip-source-port
 output
  sort +octets
  records 15
stat-definition ip-source/destination-address/ip-source-port
 report ip-source/destination-address/ip-source-port

stat-report ip-source/destination-address/ip-destination-port
 type ip-source/destination-address/ip-destination-port
 output
  sort +octets
  records 15
stat-definition ip-source/destination-address/ip-destination-port
 report ip-source/destination-address/ip-destination-port

stat-report ip-source/destination-address/ip-source/destination-port
 type ip-source/destination-address/ip-source/destination-port
 output
  sort +octets
  records 15
stat-definition ip-source/destination-address/ip-source/destination-port
 report ip-source/destination-address/ip-source/destination-port

Below is the perl script that uses the above report config file. Change the variables at the top to what you want. Currently it's set to look for the report file /etc/report.cfg. It is also set to look in the path /var/log/flow for it's log files.

#!/usr/bin/perl
# Make sure all the flow-tools binaries are in your shells path.

$cfg_file="/etc/report.cfg";
$log_path="/var/log/flow";

# Get and fix up current date.
($Second, $Minute, $Hour, $Day, $Month, $Year, $WeekDay, $DayOfYear, $IsDaylightSavings) = localtime(time);
$Month += 1;
$Year += 1900;
if($Month < 10) { $Month = "0" . $Month; }
if ($Hour < 10) { $Hour = "0" . $Hour; }
if ($Minute < 10) { $Minute = "0" . $Minute; }
if ($Second < 10) { $Second = "0" . $Second; }
if($Day < 10) { $Day = "0" . $Day; }

# Run Create file function with report name
createreport ("ip-source-address");
createreport ("ip-destination-address");
createreport ("ip-address");
createreport ("ip-source/destination-address");
createreport ("ip-source-port");
createreport ("ip-destination-port");
createreport ("ip-port");
createreport ("ip-source/destination-port");
createreport ("ip-source-address/ip-source-port");
createreport ("ip-source-address/ip-destination-port");
createreport ("ip-destination-address/ip-source-port");
createreport ("ip-destination-address/ip-destination-port");
createreport ("ip-source-address/ip-source/destination-port");
createreport ("ip-destination-address/ip-source/destination-port");
createreport ("ip-source/destination-address/ip-source-port");
createreport ("ip-source/destination-address/ip-destination-port");
createreport ("ip-source/destination-address/ip-source/destination-port");

sub createreport {
  @reportarray = (); #clear array
  open(IN, "flow-cat $log_path/$Year/$Year-$Month/$Year-$Month-$Day/ |flow-report -s$cfg_file -S $_[0] |") or die ("Can't open file. Permissions?");
    while( <IN> ) {
      if ( /# recn: / || !/^#/ ) {
        s/# recn: //;
        chomp ();
        push @reportarray, [ split "," ]; # Split elements into an array of arrays.
      }
    }
  close(IN);
  printout (); #print each report using the print subroutine
}


sub printout {
  # loop through each element in first line. count postion's over till we hit octets. exit loop.
  my $octethit=0;
  for my $titlecount ( @{$reportarray[0]} ) {
    if ( $titlecount eq "octets" ) {
      last;
    } else {
      $octethit++;
    }
  }

  # skip first array. then use octets count from above to find octets line. convert that to kb,mb,gb. write back to array.
  for my $i ( 1 .. $#reportarray ) {
    for my $j ( 0 .. $#{$reportarray[$i]} ) {
      if ( $j == $octethit ) {
        my $kilo = sprintf ("%.4f", ($reportarray[$i][$j]/1024));
        my $kiloint = $reportarray[$i][$j]/1024;
        my $mega = sprintf ("%.4f", ($kilo/1024));
        my $megaint = $kilo/1024;
        my $giga = sprintf ("%.4f", ($mega/1024));
        my $gigaint = $mega/1024;

        if ( $kiloint < 1 ) {
          $reportarray[$i][$j] = "$reportarray[$i][$j] BT";
        } elsif ( $megaint < 1 ) {
          $reportarray[$i][$j] = "$kilo KB";
        } elsif ( $gigaint < 1 ) {
          $reportarray[$i][$j] = "$mega MB";
        } else {
          $reportarray[$i][$j] = "$giga GB";
        }
          next;
      }
    }
  }

  @maxlength = (); # clear array
  for my $ii ( 0 .. $#reportarray ) {
    for my $jj ( 0 .. $#{$reportarray[$ii]} ) {
      $tmplength = length($reportarray[$ii][$jj]);
      if ( $tmplength > $maxlength[$jj] ) {
         $maxlength[$jj] = $tmplength;
      }
    }
  }

  # print to stdout all lines
  for my $q ( 0 .. $#reportarray ) {
    for my $r ( 0 .. $#{$reportarray[$q]} ) {
      my $diff = $maxlength[$r] - length ($reportarray[$q][$r]);
      print "$reportarray[$q][$r]  ";
      print ' ' x $diff;
    }
    print "\n";
  }
  print "\n";
}

This is the line from cron that emails the stats: /var/www/cgi-bin/flow-rptfmtprint.pl | /usr/bin/mailx -s "`uname -n` daily flow stats" root

Reddit!

Related stories


RSS Feed RSS feed logo

About


3com

3ware

alsa

alsactl

alsamixer

amd

android

apache

areca

arm

ati

auditd

awk

badblocks

bash

bind

bios

bonnie

cable

carp

cat5

cdrom

cellphone

centos

chart

chrome

chromebook

cifs

cisco

cloudera

comcast

commands

comodo

compiz-fusion

corsair

cpufreq

cpufrequtils

cpuspeed

cron

crontab

crossover

cu

cups

cvs

database

dbus

dd

dd_rescue

ddclient

debian

decimal

dhclient

dhcp

diagnostic

diskexplorer

disks

dkim

dns

dos

dovecot

drac

dsniff

dvdauthor

e-mail

echo

editor

emerald

encryption

ethernet

expect

ext3

ext4

fat32

fedora

fetchmail

fiber

filesystems

firefox

firewall

flac

flexlm

floppy

flowtools

fonts

format

freebsd

ftp

gdm

gmail

gnome

google

gpg

greasemonkey

greylisting

growisofs

grub

hacking

hadoop

harddrive

hba

hex

hfsc

html

html5

http

https

hulu

idl

ie

ilo

intel

ios

iperf

ipmi

iptables

ipv6

irix

javascript

kde

kernel

kickstart

kmail

kprinter

krecord

kubuntu

kvm

lame

ldap

linux

logfile

lp

lpq

lpr

maradns

matlab

memory

mencoder

mhdd

mkinitrd

mkisofs

moinmoin

motherboard

mouse

movemail

mplayer

multitail

mutt

myodbc

mysql

mythtv

nagios

nameserver

netflix

netflow

nginx

nic

ntfs

ntp

nvidia

odbc

openbsd

openntpd

openoffice

openssh

openssl

openvpn

opteron

parted

partimage

patch

perl

pf

pfflowd

pfsync

photorec

php

pop3

pop3s

ports

postfix

power

procmail

proftpd

proxy

pulseaudio

putty

pxe

python

qemu

r-studio

raid

recovery

redhat

router

rpc

rsync

ruby

saltstack

samba

schedule

screen

scsi

seagate

seatools

sed

sendmail

sgi

shell

siw

smtp

snort

solaris

soundcard

sox

spam

spamd

spf

spotify

sql

sqlite

squid

srs

ssh

ssh.com

ssl

su

subnet

subversion

sudo

sun

supermicro

switches

symbols

syslinux

syslog

systemd

systemrescuecd

t1

tcpip

tcpwrappers

telnet

terminal

testdisk

tftp

thttpd

thunderbird

timezone

ting

tls

tools

tr

trac

tuning

tunnel

ubuntu

unbound

vi

vpn

wget

wiki

windows

windowsxp

wireless

wpa_supplicant

x

xauth

xfree86

xfs

xinearama

xmms

youtube

zdump

zeromq

zic

zlib