pS-Performance Toolkit

FAQ (frequently asked questions)

pS Performance Toolkit 3.1.x FAQs

  1. Where can I ask questions or report bugs?
  2. How do I use the NPAD command line client (diag-client)?
  3. How do I use BWCTL?
  4. How do I use the NDT command line client (web100clt)?
  5. How do I use OWAMP?
  6. Can I Use a Firewall?
  7. How many ports will BWCTL need to operate effectively behind a firewall?
  8. Where can the BWCTL port values be adjusted?
  9. Can the OWAMP test port ranges be added via the GUI?
  10. What should I enter for the Communities of interest configuration question?
  11. I do not think I am a member of a community, should I put anything?
  12. What are the hardware requirements for running the pSPT?
  13. What is the recommended configuration for the hard disk of a machine running the pSPT?
  14. Does my machine have to meet the System Requirements?
  15. The colors on my Console Configuration do not match what I see on the web. Some are green already.
  16. The "Services On This Node" screen shows many services in the non-running state when first started, what is wrong?
  17. The "Services On This Node" screen doesn't have any IP addresses or hostnames for the services. Some of the services are Not Running. What is wrong?
  18. I do not see my service in the Global Set Of Services, where is it?
  19. When looking at the data display for perfAdmin BWCTL/perfAdmin OWAMP I do not see results, but I filled out information in the Scheduled Testing area. Where is my data?
  20. What is the purpose of BWCTL Limits/OWAMP Limits?
  21. How can I set limits to prevent others from overusing BWCTL/OWAMP?
  22. How many NTP servers do I need, can I select them all?
  23. Service X is listed as Not Runnning on the web interface. How can I restart?
  24. NTPD has exited/is not running on my machine, why did this happen and how can I fix it?
  25. BWCTL seems to exit immediately after starting/restarting. Why won't it stay in a running state?
  26. Can I boot from a USB key instead of a CD?
  27. How can I upgrade an 3.2RC X NetInstall?
  28. How can I backup historical OWAMP/BWCTL data?
  29. A CVE announcement was made for the current pSPT Kernel, what do I do?
  30. After creating a user on the LiveCD and trying to log in, I see Could not chdir to home directory /home/USER: No such file or directory, what is wrong?
  31. I am trying to log in with the knoppix user, and it is not working. What is wrong?
  32. During the NetInstall, I see errors about a corrupt file being downloaded. What should I do?
  33. How do I interpret the headings of each column of the "Throughput Tests" page?
  34. How do I interpret the headings of each column of the "One-Way Latency Tests" page?
  35. If I enable SSH via 'chkconfig' it does not remain across reboots, what is wrong?
  36. How do I change the MTU for a device?
  37. How do I change the SSL certificate used by the web server?
  38. I forgot to enable IPv6 in CentOS when I installed the toolkit. How do I enable it?
  39. Why is the static IPv6 address I assigned during the net-install process not configured when my host starts-up?


Q:Where can I ask questions or report bugs?
A:

For questions, send email to performance-node-users at internet2 dot edu. You may also join the mailing list by visiting https://lists.internet2.edu/sympa/info/performance-node-users. For bugs, report at http://code.google.com/p/perfsonar-ps/issues/list. State in the bug description that the issue involves the pS-Performance Toolkit.




Q:How do I use the NPAD command line client (diag-client)?
A:

The diag-client is a command line version of the NPAD diagnostic client. Instead of needing a web browser, this client runs the tests from a terminal window. The basic options are to provide a server name/address and the connection IP. The NPAD server has two ports open:

  • 8000 for HTTP traffic
  • 8001 for measurement traffic

Please connect to 8100:

[knoppix@Knoppix ~]$ diag-client HOSTNAME 8100
Using: rtt 10 ms and rate 20
Connected.
Control connection established.
port = 8003
Starting test.
Parameters based on 107 ms initial RTT
peakwin=27617 minpackets=3 maxpackets=83 stepsize=8
Target run length is 608 packets (or a loss rate of 0.16447368%)
Test 1a (11 seconds): Coarse Scan
Test 1b (11 seconds): ...
...

Connecting to the HTTP port will result in the following error:

[knoppix@Knoppix ~]$ diag-client HOSTNAME 8200
Using: rtt 10 ms and rate 20
Connected.
Protocol error: bad handshake.
Please make sure you have the latest client,  and you have the correct port number



Q:How do I use BWCTL?
A:

BWCTL (Bandwidth Test Controller) is a client/server program developed to simplify Iperf, thrulay, and nuttcp testing between hosts. At boot time, the NPTookit starts a BWCTL server process and leaves it listening on TCP port 4823. This server may then be accessed by remote BWCTL clients. Additionally, the disk contains BWCTL client applications that can be used to test to remote instances. The BWCTL server allows TCP tests with a maximum duration of 60 seconds. To run a test to a remote BWCTL server:

  • Logon to the pS-Performance Toolkit server using the root user, or other valid userid
  • Identify the remote server
  • Run bwctl -s remote-bwctl-server command to stream data for 10 seconds from the remote BWCTL server to the local instance.
  • Run bwctl -c remote-bwctl-server command to stream data for 10 seconds from the locally instance to the remote BWCTL server.
  • Results are displayed on our console or terminal window.



Q:How do I use the NDT command line client (web100clt)?
A:

The web100clt is a command line version of the NDT diagnostic client. Instead of needing a web browser, this client runs the tests from a terminal window. The basic options are to provide a server name/address and the connection IP. The NPAD server has two ports open:

  • 7123 for HTTP traffic
  • 3001, 3002, 3003 for measurement traffic

Please connect to 3001, 3002, or 3003:

[knoppix@Knoppix ~]$ web100clt -n HOSTNAME -p 3001
Testing network path for configuration and performance problems  --  Using IPv4 address
Checking for Middleboxes . . . . . . . . . . . . . . . . . .  Done
checking for firewalls . . . . . . . . . . . . . . . . . . .  Done
running 10s outbound test (client to server) . . . . .  164.00 kb/s
running 10s inbound test (server to client) . . . . . . 13.40 Mb/s
The slowest link in the end-to-end path is a a 622 Mbps OC-12 subnet
Information [C2S]: Packet queuing detected: 16.95% (local buffers)
Information [S2C]: Packet queuing detected: 67.10% (local buffers)
Server '128.193.128.237' is not behind a firewall. [Connection to the ephemeral port was successful]
Client is not behind a firewall. [Connection to the ephemeral port was successful]
Packet size is preserved End-to-End
Server IP addresses are preserved End-to-End
Client IP addresses are preserved End-to-End

Connecting to the HTTP port (or other ports) will result in the following error:

[knoppix@Knoppix ~]$ web100clt -n HOSTNAME -p 7123
Testing network path for configuration and performance problems  --  Using IPv4 address
Information: The server 'HOSTNAME' does not support this command line client



Q:How do I use OWAMP?
A:

OWAMP (One-Way Ping) is a client server program that was developed to provide delay and jitter measurements between two target computers. At boot time, the pS-Performance Toolkit starts an OWAMP server process and leaves it listening on TCP port 861. This server may then be used by remote clients. Additionally, the disk contains OWAMP client applications that can be used to test to remote instances (including a Java client and a console based application). By default, the OWAMP server sends a low-level data stream in each direction and measures the one-way delay and jitter between the two hosts. Separate measurements, one for each direction, are reported to the user at the end of the test. To run a test to a remote OWAMP server:

  • Logon to the pS-Performance Toolkit server using the knoppix or other valid userid.
  • Identify the remote server.
  • Run the owping remote-owamp-server command to make a pair of 10 second delay measurements (one in each direction) between remote OWAMP server and the local instance.
  • Results are displayed on the console or terminal window.



Q:Can I Use a Firewall?
A:

The pSPT development team recommends not limiting measurement tools to certain ports; this action may cause unexpected or unpredictable behavior. There are some caveats to enabling a firewall, namely the amount of holes that must exist for the measurement tools included on the disk:

  • SNMP MA
    • open port tcp/8065
  • PingER
    • open port tcp/8075
  • perfSONAR-BUOY
    • open port tcp/8085
    • open port tcp/8569
    • open port tcp/8570
  • Lookup Service
    • open port tcp/8095
  • BWCTL
    • open port tcp/4823
    • Edit /etc/bwctld.conf, set peer_port to a value, open the tcp port for that value
    • Edit /etc/bwctld.conf, set iperf_port, thrulay_port, and nuttcp_port to a specific range, and open the tcp/udp ports for those ranges.
  • OWAMP
    • open port tcp/861
    • Edit /etc/owampd.conf, set testports to range, open the udp ports for that range
  • NDT
    • open port tcp/3001
    • open port tcp/3002
    • open port tcp/3003
    • open port tcp/7123
  • NPAD
    • open port tcp/8000
    • open port tcp/8001
  • Apache HTTP Server
    • open port tcp/80
    • open port tcp/443
  • SSH
    • open port tcp/22
  • NTP
    • open port udp/123



Q:How many ports will BWCTL need to operate effectively behind a firewall?
A:

The pSPT development team recommends not limiting measurement tools to certain ports; this action may cause unexpected or unpredictable behavior. For instance tools such as BWCTL have two factors to consider if the ports are limited to a small subset:

  • Ports available for the regular testing infrastructure on the machine
  • Ports available for others to test to the machine

Both situations are controlled by setting values in the same configuration file on the local machine, and it can be hard to predict how many to allow. Some simple calculations can be used to determine a baseline number of ports for the first situation and are based on the parameters of the BWCTL test. Consider the following BWCTL parameters:

  • 10 Second long BWCTL tests
  • Maximum availability of 6 slots per minute

This would imply needing a total of 6 ports open. To allow for time range rounding errors we should increase this to 7 to be safe. Ideally this would work well, but there are complications due to the nature of the Linux kernel. A general behavior of the kernel is to not release a port from a previous use for up to a minute after it may be closed. This environmental consideration therefore has an impact on the above calculation. Instead of only allowing 7 ports, we should double this number to 14 to be completely safe.




Q:Where can the BWCTL port values be adjusted?
A:

The pSPT development team recommends not limiting measurement tools to certain ports; this action may cause unexpected or unpredictable behavior. If selecting a port range is still required, BWCTL has several settings defined in the bwctld.conf file that dictate which ports it may use for testing. The configuration options are:

  • iperf_port - Port range (e.g. 5001-5020) to run the iperf receiver.
  • nuttcp_port - Port range (e.g. 5021-5040) to run the nuttcp receiver.
  • thrulay_port - Port range (e.g. 5041-5060) to run the thurlay receiver.
  • peer_port - Port range (e.g. 5061-5080) to run the server processes of the above tests.

Note that the above ranges are examples, and that calculating the appropriate number of ports based on the FAQ item above is recommended.




Q:Can the OWAMP test port ranges be added via the GUI?
A:

The pSPT development team recommends not limiting measurement tools to certain ports; this action may cause unexpected or unpredictable behavior. If selecting a port range is still required, please see this section for instructions on how to alter the testing port range.




Q:What should I enter for the Communities of interest configuration question?
A:

This question can be confusing to answer for new users. The goal is to associate some loosely coupled labels to the data that the pS-Performance Toolkit disk will be making available to the larger world. Think of this step similar to assigning labels to photos or music. Some examples of valid answers are:

  • Internet2 - The data made available somehow connects the Internet2 backbone
  • LHC (CMS, ATLAS, etc.) - The disk is part of the LHC deployment structure
  • eVLBI - The disk is a part of the larger telescope community
  • MAX - A connector of member of the MAX gigapop
  • DOE-SC-LAB - US Department of Energy Office of Science Labs

Use as many community names as necessary to properly categorize the data from the installation.




Q:I do not think I am a member of a community, should I put anything?
A:

Communities are not required, but they allow other individuals and organizations to find and use your services. It is a good practice to join as many as you may think are applicable.




Q:What are the hardware requirements for running the pSPT?
A:

See this page. Note that the pSPT development team has not created hard minimum or maximum requirements - the pSPT will function on almost any form of hardware. Performance considerations do favor meeting or exceeding the minimum guidelines.




Q:What is the recommended configuration for the hard disk of a machine running the pSPT?
A:

There is only one requirement for partitioning the hard disk of a pSPT system: an ext3 partition must be available for storage. The user may configure as many partitions as they need for the pSPT disk, but for simplicity it is recommended that the entire disk be used in a single partition to allow for maximum storage resources. See also this section for hints on storage configuration.




Q:Does my machine have to meet the System Requirements?
A:

There is nothing on the pSPT that will prevent systems that do not meet the requirements from starting. Erroneous or inaccurate behavior is possible if the hardware cannot support the measurement tools.




Q:The colors on my Console Configuration do not match what I see on the web. Some are green already.
A:

If you are upgrading from a previous version of the pSPT, the colors may be green already because a particular aspect was configured previously. These do not need to be configured again.




Q:The Services On This Node screen shows many services in the non-running state when first started, what is wrong?
A:

Many of the services will be in this state because they are missing some key configuration items (e.g. from the Administrative Info). After following the configuration steps check this screen again, most should be functional.




Q:The Services On This Node screen doesn't have any IP addresses or hostnames for the services. Some of the services are Not Running. What is wrong?
A:

If the pSPT cannot grab a DHCP lease or it is not statically configured, there will be no access to the internet. Many of the services rely on knowing this information, and will therefore refuse to start unil this is corrected.

Example of a running service with a hostname:

Example of a running service, but without a hostname or ip address:

To check to be sure your installation has this information, check the /opt/perfsonar_ps/toolkit/etc/external_addresses file:

[root@localhost ~]# cat /opt/perfsonar_ps/toolkit/etc/external_addresses 
external_address=lab236.internet2.edu
default_accesspoint=lab236.internet2.edu
default_ipv4_address=lab236.internet2.edu
default_ipv6_address=2001:468:1420:3:21b:21ff:fe4e:9203



Q:I do not see my service in the Global Set Of Services, where is it?
A:

Much like DNS, the information that will populate the Global Lookup Service will take time to propagate. Please allow some time (e.g. a few hours) before your service will be fully visible.




Q:When looking at the data display for perfAdmin BWCTL /perfAdmin OWAMP I do not see results, but I filled out information in the Scheduled Testing area. Where is my data?
A:

There are several possible reasons for not seeing data:

  • Data from regular tests will take time to collect. In the case of a 4 hour testing interval, data may arrive anytime in a 4 hour window.
  • Database corruption may prevent the measurement tools from storing the results. Consult the MySQL daemon (e.g. ensure it is running and able to be accessed via the mysql -u root command
  • The measurement tools could fail to complete due to:
    • Firewalls blocking access to key ports at the source, destination, or middle connectivity of the test hosts. Attempt to run a manual test with the target tool to further explore this potential problem.
    • NTP has not fully synchronized on the source or destination host. BWCTL and OWAMP rely on stable clocks. If the clock is not stable due to system restart, lack of the NTP daemon running, or extreme clock skew (e.g. caused by using NTP servers that are not close enough to the target system), the measurement tools will fail to complete.
    • Version mismatch between source and destination systems. This occurrence is rare, and running a manual test with the target tool is the best way to further debug this problem.
    • Changes were made to the /etc/hosts file to re-direct traffic from the system's DNS name to the loopback (127.0.0.1) address. The following shows a modified version of /etc/hosts that would cause measurements to fail: 
      127.0.0.1	DNS_HOSTNAME localhost.localdomain localhost
::1		localhost6.localdomain6 localhost6
      It is suggested that /etc/hosts be similar to the following: 
      127.0.0.1	localhost.localdomain localhost
HOST_IP		DNS_HOSTNAME
::1		localhost6.localdomain6 localhost6
      The errors that may result from this modification, and will show up in the syslog output, are as follows for BWCTL: 
      bwctl: Unable to connect to HOSTNAME:4823
      And as follows for OWAMP: 
      powstream[PID]: OWPControlOpen([HOSTNAME]:861): Couldn't open 'control' connection to server: Invalid argument



Q:What is the purpose of BWCTL Limits/OWAMP Limits?
A:

These allow you to limit the influence that outside users have on your system performance. For example, to prevent your machine/network from being saturated with BWCTL tests, limit the duration and maximum bandwidth available. These screens allow a fine grained way to protect resources.




Q:How can I set limits to prevent others from overusing BWCTL/OWAMP?
A:

BWCTL and OWAMP have configurable dialog that allows the administrator to limit the resources consumed. To set the limits for BWCTL, consult this section. To set the limits for OWAMP, consult this section.




Q:How many NTP servers do I need, can I select them all?
A:

It is recommended that 4 to 5 close and active servers be used. The Select Closest Servers button will help with this decision. Note that some servers may not be available due to routing (e.g. non-R&E networks vs R&E networks - a common problem for Internet2 and ESnet servers).




Q:Service X is listed as Not Runnning on the web interface. How can I restart?
A:
  • Bandwidth Test Controller (BWCTL):
    • sudo /etc/init.d/bwctld restart
  • Lookup Service:
    • sudo /etc/init.d/lookup_service restart
  • Network Diagnostic Tester (NDT):
    • sudo /etc/init.d/ndt restart
  • Network Path and Application Diagnosis (NPAD):
    • sudo /etc/init.d/npad restart
  • One-Way Ping Service (OWAMP):
    • sudo /etc/init.d/owampd restart
  • perfSONAR-BUOY Regular Testing (Throughput):
    • sudo /etc/init.d/perfsonarbuoy_bw_collector restart
    • sudo /etc/init.d/perfsonarbuoy_bw_master restart
  • perfSONAR-BUOY Measurement Archive:
    • sudo /etc/init.d/perfsonarbuoy_ma restart
  • perfSONAR-BUOY Regular Testing (One-Way Latency):
    • sudo /etc/init.d/perfsonarbuoy_owp_collector restart
    • sudo /etc/init.d/perfsonarbuoy_owp_master restart
  • PingER Measurement Archive and Regular Tester:
    • sudo /etc/init.d/PingER restart
  • SNMP Measurement Archive:
    • sudo /etc/init.d/snmp_ma restart



Q:NTPD has exited/is not running on my machine, why did this happen and how can I fix it?
A:

NTPD may exit if the hardware clock on the host is too far off from the true time to make a difference. To skip the clock ahead to the correct time, try these commands:

[knoppix@Knoppix init.d]$ sudo /etc/init.d/ntp stop
Stopping NTP server: ntpd.
[knoppix@Knoppix init.d]$ sudo ntpdate owamp.newy.net.internet2.edu owamp.wash.net.internet2.edu
Looking for host owamp.newy.net.internet2.edu and service ntp
host found : eth-0.nms-rlat.newy32aoa.net.internet2.edu
Looking for host owamp.wash.net.internet2.edu and service ntp
host found : eth-1.nms-rlat.wash.net.internet2.edu
27 Jan 13:42:51 ntpdate[14891]: adjust time server 2001:468:9:12::16:34 offset -0.001660 sec
[knoppix@Knoppix init.d]$ sudo /etc/init.d/ntp restart
Stopping NTP server: ntpd.
Starting NTP server: ntpd.

If NTPD continues to exit on a periodic basis, there may be a hardware failure. Consult the machine's BIOS to see if there may be problems with the hardware clock or internal battery.




Q:BWCTL seems to exit immediately after starting/restarting. Why won't it stay in a running state?
A:

BWCTL relys on NTP (Network Time Protocol) to have an accurate representation of time for measurements. BWCTL (and ntpd) will simply exit if the system clock is too far from the recognized time. To check the status of the ntpd daemon:

[knoppix@Knoppix ~]$ ps axw | grep ntpd
 5146 ?        Ss     0:00 /usr/sbin/ntpd -p /var/run/ntpd.pid -u 115:121 -g

If ntpd is not running, you will get nothing back:

[knoppix@Knoppix ~]$ ps axw | grep ntpd
 5140 pts/0    R+     0:00 grep ntpd

To bring your system clock back up to date, try the following steps:

sudo /etc/init.d/ntp stop 
sudo ntpdate owamp.newy.net.internet2.edu
sudo /etc/init.d/ntp start

To check your system clock on the pSPT, try the following command (after restarting ntpd):

[knoppix@Knoppix ~]$ ntpq -p -c rv
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
*chronos.es.net  .PPS.            1 u   30   64    1   25.016    7.335   0.170
+navobs1.oar.net .USNO.           1 u   29   64    1    7.299    5.607   0.065
+tick.usno.navy. .USNO.           1 u   28   64    1   40.144    7.690   2.046
-2001:468:1:12:: 130.207.244.240  2 u   27   64    1   27.212    5.913   0.052
-2001:468:2:12:: 64.57.16.34      2 u   26   64    1   30.335    5.501   0.047
assID=0 status=0644 leap_none, sync_ntp, 4 events, event_peer/strat_chg,
version="ntpd 4.2.2p4@1.1585-o Sun Nov 22 16:42:02 UTC 2009 (1)",
processor="i686", system="Linux/2.6.27.37-web100", leap=00, stratum=2,
precision=-20, rootdelay=25.016, rootdispersion=946.806, peer=21397,
refid=198.124.252.90,
reftime=cf7c5bc9.8f99a127  Fri, Apr 23 2010 13:47:53.560, poll=6,
clock=cf7c5bf1.8e0ffeba  Fri, Apr 23 2010 13:48:33.554, state=4,
offset=7.335, frequency=-47.691, jitter=1.113, noise=2.593,
stability=0.020, tai=0

If you find that your clock is stopping on a regular basis, the internal battery of your server may be failing. Consult your server user's manual or on-line references for more information.




Q:Can I boot from a USB key instead of a CD?
A:

Yes, the CentOS project, recommends the method found here.

Additionally, there are tools such as UNetbootin available to migrate an ISO to a USB key.




Q:How can I upgrade an 3.2RC X NetInstall?
A:
  • Download http://software.internet2.edu/rpms/i386/RPMS.main/Internet2-repo-0.2-5.noarch.rpm
  • Install the RPM, e.g. sudo rpm -ivh Internet2-repo-0.2-5.noarch.rpm
  • Edit /etc/yum.repos.d/Internet2-web100_kernel.repo and set enabled = 1 
    # Name: Internet2 RPM Repository
# URL: http://software.internet2.edu
[Internet2-web100_kernel]
name = Internet2 Web100 Kernel RPM Repository - software.internet2.edu - main
mirrorlist = http://software.internet2.edu/web100_kernel/rpms/mirrors-Internet2-web100_kernel
enabled = 1
protect = 0
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-Internet2
gpgcheck = 1
  • Run yum update
  • Reboot



Q:How can I backup historical OWAMP/BWCTL data?
A:

These steps will get rid of all data older than 3 months (adjust the 'maxmonths' if you want to allow more to stay in the db), and store them as SQL dumps on the host. Here are the steps to clean the database on the throughput machine:

sudo mkdir -p /var/log/BACKUP/bwctl
sudo /opt/perfsonar_ps/perfsonarbuoy_ma/bin/clean_pSB_db.pl \
--mysqldump-opts="--skip-lock-tables" --dbtype=bwctl --maxmonths=3 \
--owmesh-dir=/opt/perfsonar_ps/perfsonarbuoy_ma/etc/ \
--dumpdir=/var/log/BACKUP/bwctl

Run the following command on latency machine to archive data older than 30 days (adjust the 'maxdays' to keep more):

sudo mkdir -p /var/log/BACKUP/owamp
sudo /opt/perfsonar_ps/perfsonarbuoy_ma/bin/clean_pSB_db.pl \
--mysqldump-opts="--skip-lock-tables" --dbtype=owamp --maxdays=30 \
--owmesh-dir=/opt/perfsonar_ps/perfsonarbuoy_ma/etc/ \
--dumpdir=/var/log/BACKUP/owamp



Q:A CVE announcement was made for the current pSPT Kernel, what do I do?
A:

The perfSONAR-PS development effort subscribes to all major outlets that will announce kernel CVEs. In the event that a CVE is announce that directly effects operation of the pS Performance Toolit, the following steps will take place:

  • Announcements regarding the CVE will be posted to the performance-node-users mailing list, more information on the mailing list is available here: https://lists.internet2.edu/sympa/info/performance-node-users
  • A timeline will be relayed regarding availability of new kernels.
  • The CentOS project will make the patched kernel available first, and it will be available through the yum repositories on the toolkit before the perfSONAR-PS project is able to apply the web100 patches.
    • It is strongly suggested that pS Performance Toolkit users utilizing the NetInstall option upgrade immediately. Note this will break support for the NDT and NPAD tools, but use of measurement tools is secondary to security. Run the following command: 
      sudo yum update
    • pS Performance Toolkit users that are utilizing the LiveCD option can either continue using their instance, or shut it down depending on the severity of the CVE.
  • The perfSONAR-PS project will release a web100 patched version of the most recent kernel, and make these available through the yum repo. Announcements will be made again to the performance-node-users list.



Q:After creating a user on the LiveCD and trying to log in, I see Could not chdir to home directory /home/USER: No such file or directory, what is wrong?
A:

The Storage Section of the pS Performance Toolkit Quickstart Guide notes that if you make changes to the backing store during configuration - you should reboot immediatly. Adding users via the User Management commands before rebooting will result in home directory errors.

To correct this problem, delete and re-add the user account.




Q:I am trying to log in with the knoppix user, and it is not working. What is wrong?
A:

Version 3.2 of the pS Performance Toolkit no longer adds a knoppix user by default. If you have upgraded from version 3.1.x then you will still have this user on your system. Please use the root user as the default way to log into the system.




Q:During the NetInstall, I see errors about a corrupt file being downloaded. What should I do?
A:

During the NetInstall, you may see some errors about a corrupt file being downloaded along with buttons like Reboot and Retry. This happens if it fails to download an RPM from a mirror, which can happen for numerous reasons. Usually, that error can be solved by hitting Retry. You may have to hit that multiple times depending on which mirrors the install is trying to download the RPM from.




Q:How can I upgrade from the 3.1.x LiveCD to the 3.2.x NetInstall?
A:

There is not an automatic way to migrate existing data from a legacy LiveCD deployment to the hard disk environment. The following steps can be used as guidelines.

  1. Boot into 3.1.x, and backup the previous configuration/data.
    • MySQL: Dump all of the mysql databases (bwctl, owamp, cacti, pingerMA, and mysql), and store these on a different machine. 
      mysqldump -u root owamp > owamp.sql
mysqldump -u root bwctl > bwctl.sql
mysqldump -u root cacti > cacti.sql
mysqldump -u root pingerMA > pingerMA.sql
mysqldump -u root mysql > mysql.sql
    • Cacti (If using Cacti): Copy the contents of /var/lib/cacti to a different machine.
    • Host Configuration: Copy the file /mnt/store/configs.tbz to a different machine.
  2. Shutdown the machine, insert the NetInstall image (N.B. a different machine can be used also). Setup the NetInstall, do not configure via the steps in the quick start (e.g. do not run /opt/perfsonar_ps/toolkit/scripts/nptoolkit-configure.py or configure things via the web interface).
  3. Copy the contents of /var/lib/cacti back to this machine.
  4. The following step may bot be necessary, but will create the necessary databaes on the new machine: 
    mysqladmin -u root create owamp
mysqladmin -u root create bwctl
mysqladmin -u root create cacti
mysqladmin -u root create pingerMA
mysqladmin -u root create mysql
  5. Restore the database dumps into mysql (the complete dumps will preserve the tables etc. as they were on the old host - there were no database schema changes between versions) 
    mysql -u root owamp < owamp.sql
mysql -u root bwctl < bwctl.sql
mysql -u root cacti < cacti.sql
mysql -u root pingerMA < pingerMA.sql
mysql -u root mysql < mysql.sql
  6. Copy the config tarball to somewhere on the machine.
  7. Download the following script: http://anonsvn.internet2.edu/svn/perfSONAR-PS/trunk/perfSONAR_PS-Toolkit/scripts/upgrade_3.1.x.sh
  8. Edit the script:
    • Set STORE_LOCATION to be the directory of the tarball
    • Set the OLD_CONFIG_TARBALL variable to the location of the tarball, use the STORE_LOCATION variable (e.g. $STORE_LOCATION/configs.tbz)
  9. Run the script.
  10. Download the following script: http://anonsvn.internet2.edu/svn/perfSONAR-PS/trunk/perfSONAR_PS-Toolkit/scripts/upgrade_fix_permissions.sh
  11. Run the script.
  12. Networking configuration for the new machine vs the old machine may have been altered during the configuration move, particularly with respect to the numbering of the network interfaces. The networking configuration tool: 
    sudo /usr/sbin/system-config-network-tui
    Can be used to verify that the settings for DHCP or Static IP address are configured on the correct interface.
  13. Open and edit /opt/perfsonar_ps/toolkit/etc/discover_external_address.conf, be sure the external_interface= entry reflects the current externally facing interface.
  14. Reboot



Q:How do I interpret the headings of each column of the "Throughput Tests" page?
A:

  • First Host - The hostname of the first host in the test.
  • First Address - The IP address of the first host in the test.
  • Second Host - The hostname of the second host in the test.
  • Second Address - The IP address of the second host in the test.
  • Protocol - Indication that a TCP or UDP is being performed.
  • Duration - The length of the bandwidth test
  • Window Size - If a window size was selected for TCP tests, it will be displayed here. If a window was not selected (e.g. TCP auto-tuning) nothing will appear.
  • Bandwidth Limit - If a UDP test was selected, it is possible to limit the total bandwidth that can be consumed for a test. If the test is TCP or no limit was entered, this will be blank.
  • Bi-Directional - Indication that both directions of the test are available. OWAMP tests each direction, and when one direction has failed it may indicate:
    • Firewalls are preventing the test from completing
    • NTP is not synchronized on one of the hosts
    • Either host does not have available test slots
    • The results of the test could not be retrieved
  • Line Graph - A drop down that features different time ranges. When clicked a graph featuring connected points will appear.
  • Scatter Graph - A drop down that features different time ranges. When clicked a graph that only displays data points will appear.




Q:How do I interpret the headings of each column of the "One-Way Latency Tests" page?
A:

  • First Host - The hostname of the first host in the test.
  • First Address - The IP address of the first host in the test.
  • Second Host - The hostname of the second host in the test.
  • Second Address - The IP address of the second host in the test.
  • Bi-Directional - Indication that both directions of the test are available. OWAMP tests each direction, and when one direction has failed it may indicate:
    • Firewalls are preventing the test from completing
    • NTP is not synchronized on one of the hosts
    • Either host does not have available test slots
    • The results of the test could not be retrieved
  • Graph - A drop down that features different time ranges. When clicked a graph will appear.




Q:If I enable SSH via 'chkconfig' it does not remain across reboots, what is wrong?
A:

The initialization of certain services such as SSH or perfSONAR daemons, normally controlled by tools such as 'chkconfig', is managed by the pS Performance Toolkit web interface. The proper way to enable and disable services is to visit https://[host]/toolkit/admin/enabled_services to manage service startup.


Q:How do I change the MTU for a device?
A:

You can view the MTU of your network devices by executing the /sbin/ifconfig command.

To temporarily change the MTU for a device, you use the ifconfig command and specify the device and the new MTU. For example: 

ifconfig eth0 mtu 9000 up

To make these changes permanent you need to modify the specific devices configuration file. These files are in /etc/sysconfig/network-scripts/ and have names like ifcfg-eth0 for the device eth0 and ifcfg-eth1 for eth1.

  1. For example you could add the line MTU="9000" for IPv4 or IPV6_MTU="9000" for IPv6 to /etc/sysconfig/network-scripts/ifcfg-eth0.
  2. After making the changes you need to restart the network services by running the command 'service network restart' as root.




Q:How do I change the SSL certificate used by the web server?
A:

The toolkit by default generates a self-signed SSL certificate that it configures for use with the Apache web server. Some users may desire to replace this certificate with a certificate signed by a certificate authority (CA).

You may also need to replace the certificate due to a problem sometimes encountered with browsers not accepting the self-signed certificate. You may see an error like the following: 

    HOST uses an invalid security certificate.
    The certificate is not trusted because it is self-signed.
    The certificate is only valid for localhost.localdomain
    (Error code: sec_error_untrusted_issuer)

You can find instructions for installing a new certificate in Apache here.




Q:I forgot to enable IPv6 in CentOS when I installed the toolkit. How do I enable it?
A:

It is recommended that you always enable IPv6 during the CentOS installation portion of the toolkit setup. If you did not enable it, then you can do so with the following steps:

  1. Login to the toolkit as a user capable of running sudo
  2. Run sudo and enter your sudo password
  3. Open the file /etc/modprobe.conf in a text editor and remove the following lines: 
    alias net-pf-10 off
alias ipv6 off
options ipv6 disable=1
  4. Restart the host
  5. You can now assign an IPv6 address.




Q:Why is the static IPv6 address I assigned during the net-install process not configured when my host starts-up?
A:

When you perform the net-install of the toolkit, you will be prompted twice to enter networking information by CentOS. The first time is to define the networking to be used for downloading required packages. The second prompt is later in the installation and defines what will be configured on the host post-installation. It is a known CentOS behavior that IPv6 information entered at the first prompt is not automatically filled-in at the second prompt. This can be confusing because the IPv4 information does get automatically filled-in. If you do not manually enter the IPv6 information a second time, then your host will not have the address configured post-installation. You will have to manually assign the address if this happens.