Stop the Bleeding. Heal the IT Pain.

This blog showcases a recent project that seemed to be an easy IT fix, unearthing several other IT issues. Do you fix all the issues at once? Do you just wait until the other issues break and then fix them? Or, do you find an IT partner who will do the work well, not nickel and dime you, and provide answers for avoiding problems down the road? You decide. 

Topics covered:

• The major disruption that kept remote employees from accessing the network
• Simple IT fix leads to the uncovering of several hidden problems
• The fix to these problems
• The real value of working with an IT Partner who does not just fix the immediate problem

Every company needs to protect their customer and financial data and to do this. This Arizona business team had implemented Cisco firewalls. While the firewalls were working, remote employees were being denied access to the network.

To complicate matters further, firewall licenses had expired, making it impossible to get into the firewall configuration to provide secure remote access to the network.

This seemed like a simple fix. Either renew the licensing on the older equipment OR move to a newer next-generation firewall.

After some due diligence, it was determined that it would cost roughly the same amount of money to renew the existing firewall licenses as it would cost to purchase new hardware, complete with all the necessary functionality and security services, on a more intuitive and easier to use interface. Additionally, subsequent license renewals on the new hardware would cost 75% less than renewing existing licenses annually. The team quickly agreed to update their firewall.

The overall project objective would be a simple cutover from the existing firewalls to a new Meraki Firewall Gateway. But what happened next was a chain of hidden problems that required immediate action to restore appropriate access to critical network resources.

• Cables and devices in the server room were connected in such a way that caused several network loops.
• Traffic in the loop overloaded the network connections on the host that housed the servers.
• Host housing the servers were overloaded with corrupt network traffic, causing the host to crash.
• This, in turn, caused all the Virtual Servers (located on that host) to crash, affecting the host’s stability upon restart.
• When the Virtual servers crashed, performance and stability problems made it difficult for remote workers to authenticate and connect to the network resources. (Yikes!)
• When the virtual servers went down, the Dynamic Host Configuration Protocol (which automatically provides the host with your IP) prevented users from getting proper IP addresses to access the network.
• Misconfiguration issues caused by an outside vendor caused voice call connection issues, making it difficult to make or receive calls.
• When all was said and done, the old core network switch could not handle what was happening on the network, and it too crashed.

What a mess! Many IT partners would have fixed the firewall and charged for additional fixes. But, not Blue Fox Group.

To remediate the chain reaction of issues, the Blue Fox Team addressed and fixed the issues:

• Traced cabling and reworked the configurations to eliminate all network loops.
• All virtual servers were migrated to a temporary host, allowing the BFG team to update the older host firmware and hardware components to provide a more stable operating environment.
• New servers were built to replace the functionality of the former virtual machines.
• Remote access was restored by authenticating the newly built server.
• Wireless connectivity was restored, and a new Meraki Gateway was cut over.
• They wrapped up the project by restoring switch functionality and replaced outdated switches to ensure the network performed well.

Often one IT issue leads to another. Thank goodness this company did not work with an IT company that fixed the contracted problem, only to be called back again to fix another at an expensive price tag. In addition to taking care of 3rd party vendor issues and fixing all the problems at one time, Blue Fox Group provided the management team at this organization with “next step” recommendations for improving the quality of the network and its service, increasing redundancy, faster network speeds, and a better user experience. Most importantly, they gave this company options for avoiding future IT surprises. Ultimately, a good IT partner will provide your team with IT controls and transparency into the network’s health, preventing unexpected IT issues.

Technology alone is not enough. A trustworthy IT professional, acting as your partner, diagnosing business pain (and not just IT pain) can reduce your need for frequent technical support and refocus your efforts on deciding how IT can support your business objectives and outcomes.