Performance and Usability Analysis
Our objectives for this project included:
- Relocate all network equipment to a secure data center facility
- Reduce support costs by leveraging a single point of access to all essential network equipment
- Increase network scalability for future growth
- Increase remote manageability of servers
As part of the project, United’s core switching infrastructure was upgraded. Previously, United used Nortel Baystack 450 switches which provided a 2.5 Gbps switch fabric capacity and a 3 Mpps forwarding rate. The switching fabric is the native internal packet switching speed of the switch. Based on our research, the Cisco Catalyst 3750G 24-port switch was recommended to United. This switch provides a greatly improved 32 Gbps switching fabric and 38.7 Mpps forwarding rate. In addition to the performance enhancements offered by the 3750G switches, they also add management and scalability to United’s core network that was not offered before by the existing Baystack switches. The new 3750G switches utilize the familiar Cisco command line interface for management and allow switches to be stacked for future growth.
From a usability standpoint, remote access to the servers was improved. In the past, technicians had to move between buildings and various different monitors in order to access servers when troubleshooting. The newly installed IP based KVM breaks that barrier and allows United’s technicians to access servers, routers, and switches via a web-based interface from anywhere in the world. Installing the new IP based KVM also allowed for the clearing of bench space where a dedicated monitor, mouse, and keyboard used to reside for each server.
In addition to remote access, the support time has been reduced as part of this project. From our own experience it has taken up to fifteen minutes in order to just trace a single unlabeled cable from one end of a device to another. With a properly organized and labeled cabling system, United is able to eliminate that unnecessary burden when working to resolve an issue and quickly get to the source. Due to the time saved by cable documentation and server centralization, United’s IT staff has been able to reduce amount of man-hours dedicated to performing cable traces.
Successes and Failures
Like any project, we had our share of successes and failures. Fortunately, we were able to learn from all of our failures and eventually overcame them with a solution.
Our largest failure was when we purchased and attempted to use the wrong fiber adapters for the new Cisco switches. Our goal was to provide them with the ability to have gigabit connections in the future, while maintaining current Fast Ethernet connections. However, due to an oversight while ordering the fiber optic adapters it was not realized that they would only run at gigabit speeds. This led to the fiber connections not working correctly since the current network only supported 100 Mbps. Fortunately we were able to exchange the adapters for the correct ones and install them.
Another failure we had was switch configurations. Significant time was spent on preparing the switch configurations for the move; however when the White network was transitioned, it was determined the VLAN configuration had not been saved properly. This error was realized after spending a considerable amount of time troubleshooting the switch configurations. Once the problem was located, it was quickly resolved.
We had not planned Daylight Savings Time when the date was set for our project. Due to this fact, an hour of our already short maintenance window was lost for the tier 3 equipment move. This reduced our six hour window down to five hours.
On initial use, the slide-out monitor appeared to be a failure. After researching the problem, it was determined there was a firmware upgrade for the IP based KVM was necessary to fix the issues we were experiencing. After updating the firmware, the device worked flawlessly.
We felt our cable management solution was a success. The labels we selected adhered to the cables properly. They also allowed us to include enough information on each label for proper documentation. This solution proved immensely useful in troubleshooting cabling issues as the project progressed. The Black Box raceways provided sufficient space and organization to route and hide all the data and power cabling in the new racks.
The move schedule that we prepared was also another success. With the proper planning and foresight we were able to arrange moves in such a fashion that minimized customer downtime. The schedule included leeway in the event of any troubles. This foresight was utilized during the Tier 2 relocation. The schedule also allowed United’s technicians to minimize the amount of time spent onsite during the relocation.
Another success was the move of the Blue network. Several difficulties had been anticipated, but due to our careful planning, we didn’t end up experiencing any of them during the move. The Blue network came up with minimal hassle, and customers on that network only experienced negligible downtime.
What we learned
Throughout the course of this project, we learned many important skills, mostly relating to project management and intraorganizational communication skills. We anticipated learning more technical skills, while instead we learned a great deal more about organizational communication and leadership.
Project management was a crucial component to the success of our project. Within each of the four different phases there were many small tasks that had to be completed on time. Most of these tasks were spread out over a large group of individuals within United in addition to our group members, so keeping track of the progress was challenging. Fortunately we were able to utilize delegation skills that we have learned throughout our college career and break things into smaller, more manageable pieces.
Another important aspect we learned was the necessity of proper documentation and organization. It is easy when installing new equipment to not take the time to label the cables or organize the wiring, because one needs to simply just get it working. However, by taking a little time to do this, you can save a lot of time in the future. Documentation for is a key to the troubleshooting process. By documenting switch configurations, creating a network layout diagram, and properly labeling cables, we have established a system that will allow for quick troubleshooting of problems in the future.
Intraorganizational communication skills were a definite requirement for this project. It was tough for some members of the team to arrive at United and then have to tell United employees what needed to be done. However, we were able to utilize the concept of lateral leadership, which greatly improved the ability to work along side the United staff to complete the project. We learned that it is important to be able to lead and follow at the same time.
Throughout the project our group was able to experience consulting on a first hand basis. Rather than just reading how Weiss thought it should be done, we were able to hone our skills directly with a client. It was a new experience to work with someone else’s budget and having to adhere to its guidelines. It was also a learning curve to first understand their network in all its detail before starting to recommend new solutions. Overall, it was a positive experience for each group member.Recommendations
This project was just one of the beginnings of the network upgrades that United is beginning to face with the advent of all of its new services.
We recommend that United continues to upgrade the rest of its switched network. With internet subscribers reaching the thousands and more bandwidth being utilized on a global scale, it is imperative that United work to resolve throughput issues within its own network. We believe the best solution to this would be to continue to deploy more powerful switches throughout the network, such as the Cisco Catalyst 3750G. These switches will bring greatly increased performance, management, and the ability to scale easily.
Further, we recommend the upgrade of long-distance backhauls between the core switches and distribution layer switches. Gigabit backhauls would provide these key links with the bandwidth needed to sustain increased traffic demands and allow for expansion.
We also recommend that United continues to utilize the cable management system we established. As noted earlier, this is crucial to troubleshooting and will save United much time in the future.
Finally, we recommend working towards redundancy in both the Blue and White networks. These networks should be able to successfully failover to each other in an emergency situation. A majority of United’s equipment is already configured to do this, but one server in particular is still a weak link in this setup. The current Cisco Secure server used the authenticate DSL, Wireless, and Dial-up customers is only connected to the White network and not the Blue network. In the event of a failure of the White network, customers on the Blue network trying to authenticate will also be affected. In order to alleviate this problem, an additional interface should be added to this device so it will be available to either network.
Thank You
This project would not have been possible without help from United Telephone Association, Inc. We would like to take this opportunity to thank Craig Mock, general manager of United for approving of this project. We also thank Mike Salmans, IT Manager, for helping to establish this project, and work to ensure we had the right resources when we needed them. We also thank the rest of the IT department at United for being available to help us with the move. We needed expert help when moving several pieces of equipment and their knowledge made this possible. We wish United the best of luck in the future and hope it continues to exceed its expectations for customer service.