The perfSONAR Performance Toolkit (pS-Performance Toolkit) is a customized distribution of the CentOS operating system. The software consists of an ISO disk image that can be used in one of two ways: burn the image to a disk (recommended); or mount the ISO file system and run as a virtual machine (for example, under VMware, QEMU or xen). After performing either step, the result is a fully-configured system capable of running and using many advanced network tools and methodologies, serving either as a testing point or a means to solve performance issues.
On April 7th 2014, the perfSONAR project received notice from the CentOS of a CVE related to the OpenSSL package: https://rhn.redhat.com/errata/RHSA-2014-0376.html
This alert was triggered by the recent exploit entitled "heartbleed",
which targets TLS activity. perfSONAR is recommending that all users run
yum update to
download the latest packages from the CentOS repositories. Additionally, we recommend re-generation of the
host's certificate information after the packages are updated, as it was generated with the vulnerable
implementation of OpenSSL. The following steps will accomplish this goal:
- # build a new key
/usr/bin/openssl genrsa > /etc/pki/tls/private/localhost.key
- # create a new self-signed cert
cd /etc/pki/tls/certs make testcert
- # restart apache
sudo /etc/init.d/httpd restart
If you have installed a customized certificate, refer to instructions specific to that process.
New versions (3.3.2-2) of the LiveCD and LiveUSB product have been generated, and are available here.
Some of you may be aware of a recently highlighted risk of NTP amplification (http://www.symantec.com/connect/blogs/hackers-spend-christmas-break-launching-large-scale-ntp-reflection-attacks), and in some cases others may have already had a host that was used in an attack. A sample notice received may look something like this:
You are running a public NTP server that participated a very large-scale attack against a customer of ours today, generating UDP responses to spoofed requests with bogus timestamps that claimed to be from the attack target. Your server was particularly active in the attack, sending a significant percentage of the attack traffic we saw.
Please consider reconfiguring your NTP server in one of these ways:
- To only serve your customers and not respond to outside IP addresses. If your NTP server runs as a standalone installation, setting the service to ignore all queries would work well for this. With ntpd, that can be done with "restrict default ignore" in /etc/ntp.conf; other servers should have a similar configuration option. A firewall rule to block UDP to the public IP address on port 123 would also work for this.
- To rate-limit responses to individual source IP addresses
- To limit queries to TCP-only
- To ignore particularly unlikely queries, such as those representing dates far in the future or past
- To limit the size of allowed responses
Starting with the 3.3.2 release of the pS Performance Toolkit, configuration changes have been implemented to correct the risk of NTP amplification. We highly recommend updating hosts to this version to mitigate the risk of being a part of future protocol abuses. For those that have not updated to the latest version, it is possible to make manual modifications to the local /'etc/ntp.conf' file to accomplish the same goal:
# by default act only as a basic NTP client restrict -4 default nomodify nopeer noquery notrap restrict -6 default nomodify nopeer noquery notrap # allow NTP messages from the loopback address, useful for debugging restrict 127.0.0.1 restrict ::1
Additional restrict lines can be added to allow trusted subnets, e.g.:
restrict a.b.c.d mask 255.255.0.0 nomodify notrap nopeer
Any changes to NTP should be followed by a restart of the service. More information on protection for hosts (and routing devices) can be found here:http://www.team-cymru.org/ReadingRoom/Templates/secure-ntp-template.html
pS-Performance Toolkit 3.3.2 released
pS-Performance Toolkit Recommendations for Use with Firewalls
The pS Performance Toolkit development team, working closely with the NTAC Performance Working Group, has produced a comprehensive guide to configuring site and host firewalls for use with the measurement tools. This information is provided because:
- Well known ports will make it easier for sites that are using firewalls to specify the requirements of measurement on the host itself, and all network devices in the middle
- Sites that are not using firewalls will be able to test to firewalled sites (as well as firewalled sites testing to them) and be guaranteed of success. Previous attempts in this space could have involved sites choosing different 'ranges' for their tools, and the tools did not effectively negotiate on a strategy that would lead to working measurement.
More information is available here.
When serving as a measurement point, a use case adopted by Large Hadron Collider (LHC) affiliated Tier 2 sites in the United States, the disk becomes a part of a global registry of perfSONAR tools. Affiliated sites and other interested parties may perform any number of tests (on demand, or in some cases regularly scheduled) to the installation including:
- BWCTL - Bandwidth Test Controller
- Cacti - Network data polling and graphing
- NDT - Network Diagnostic Tool
- NPAD - Automatic diagnostic server for troubleshooting end-systems and last-mile network problems
- OWAMP - One-way Ping
- perfSONAR-PS Services
- Lookup Service - perfSONAR registration and discovery
- PingER Measurement Archive and Measurement Point - Perform and archive latency measurements
- perfSONAR-BUOY Measurement Archive and Collection Framework - Perform and archive bandwidth and one way latency measurements
- SNMP Measurement Archive - Archive SNMP data
When used in the role of a network debugging infrastructure, the various tools can be used to methodically find, diagnose, and aid in the correction of network performance issues. Additional supporting components include:
- Apache2 - Web server with supporting modules and extensions
MySQL - Relational database management system
- NTP - Network Time Protocol
installation and disk contents
The original idea for the pS-Performance Toolkit came from the Network Performance Workshop series sponsored by Internet2. The old format of this workshop consisted of a day of tool installation (e.g. installing the Web100-enhanced kernel and other performance tools) followed by some time to test and use them. This particular format did not allow enough time to get to the primary reason many participants choose to sign up: solving network performance problems.
To lower the bar on this complex topic, this product offers a complete network performance suite pre-installed (and in many cases pre-configured) that is ready to be dropped directly into a network operations center (NOC) of any size. With the integration of perfSONAR tools, the results of both passive and active measurements can be shared in a federated environment with your peers enabling the early detection of performance issues.