More than a year before the COVID-19 Pandemic forced many workers to WFH, this leading management consulting company in Connecticut decided to implement a WFH directive in order to streamline costs. Now that the COVID-19 virus has forced almost all organizations to temporarily close their offices in the midst of social distancing, a WFH solution is becoming the ‘new normal’.
Initially, in examining ways to be more efficient, their Management recognized most of its consultants were working routinely at client sites many days during each week. The cost of maintaining a central office that was not used much of the time was significant. Furthermore, their expensive office space did not have room for expansion, specifically to accommodate new associates joining the firm! Moving to a bigger office would be very expensive and require a lot of effort. This, along with the rapid growth of the firm, required a long-term solution, and after exhausting many alternatives, they ultimately decided to adopt a WFH model. This would provide the flexibility they needed to adapt to their developing customer facing requirements, and it would be more cost effective.
From a technology standpoint, relocating employees to work from their homes was straightforward. These management consultants already had laptops and mobile phones to support their field work, so they had the tools needed for a home office. However, there was one significant challenge to be resolved.
Management struggled with where to locate the company’s servers. Though many of the firm’s workloads had been moved to the Cloud for applications like O365, the firm still had several servers that were required to deliver important proprietary financial and database applications. The management of the firm was not comfortable placing these critical workloads in the Public Cloud because of their stringent security requirements. They also believed the cost of moving some of their larger database applications to the Public Cloud would be much more expensive.
An alternative was to move the servers to the home of one of the senior partners, but this option did not have an environment with the required enterprise quality power, air conditioning, and the necessary security measures. They also required accessibility at any time of the day or night.
They realized they needed to find a colocation partner that could provide:
- Environmental systems to assure proper temperature and humidity at all times
- A redundant Power Supply backed up by redundant Air Conditioning, UPS, and Generator systems
- A purpose-built facility with 24/7/365 secure access and continuous video monitoring
- Remote Hands services to assist with tasks like rebooting servers when necessary
- A colocation partner with the flexibility to provide a tailored service including a facility for occasional meetings of some the firms’ employees
The firm considered a number of colocation service providers and selected Blue Hill/CAPS for the following reasons:
- A stellar record of over 20 years without service disruptions caused by unscheduled power outages
- SSAE 18 (SOC 1 Type 2) secure data center with 24/7/365 access
- Cost effective colocation solution
- Tailored solution including Remote Hands services and use of meeting rooms as needed
- Assistance procuring an IP based telephone system
In the time since the decision was made to partner with Blue Hill/CAPS, this consulting firm has realized the following benefits:
- Savings of approximately 50% of the cost of its former operation
- Improved flexibility and ability to add staff as needed
- More secure servers with more resiliency, better power, internet capabilities, and environmental controls
- Many of the recurrent IT infrastructure support tasks are now handled seamlessly
- An upgraded telephone system saved the firm money and improved functionality
As many organizations employ WFH on a much bigger scale in response to COVID-19, planning ahead and especially contingency planning can have very tangible business benefits in working through this time of great change.
Connecticut has big data backup requirements. With a large concentration of financial service, legal, healthcare, education, and government organizations generating many terabytes of valuable data, the state is frequently targeted by cyber criminals. It also experiences a significant number of natural emergencies whether they are caused by snowstorms, hurricanes, floods, fires or any number of totally unexpected circumstances.
Timely data backups are essential so operations can resume as soon as possible after a disruption. But just having a current copy of your data does not guarantee you will be able to successfully recover. Successful restoral, in case of an actual disaster, requires having adopted and successfully managing a dependable data backup and recovery solution.
It also is important to test the process periodically. There are many examples of organizations that were unable to recover their backup successfully when they were under the gun. In fact, a recent survey by Hanover Research of two hundred large and mid-sized companies in the U.S found that 64% were unable to recover their data when they needed to; even though they had a backup system in place.
There are a wide variety of available backup technologies ranging from inexpensive consumer products that backup an individual computer to sophisticated enterprise level solutions that are designed to backup thousands of servers across a large distributed network with the ability to restore one or more Virtual Machines (VMs) or an individual file in minutes.
There are five variables to consider when selecting an appropriate data backup and recovery solution-
- Labor Requirements
Effectiveness means how likely will it be you recover your required data in a timely manner? In general, the faster the recovery, the more expensive the solution. Recovery Point Objective (RPO) and Recovery Time Objective (RTO) define how much data you are willing to risk and how fast you expect to recover.
Backup solutions vary in the amount of manual effort required. Some systems are very labor intensive. This not only adds expense, it also can impact effectiveness if individuals fail to perform their assigned backup duties. The additional cost of automated solutions is typically justified by higher availability.
Different backup solutions provide different levels of data security. A backup solution that easily can be breached would add insult to injury. Sophisticated systems employ encryption and other technologies to mitigate security risks.
The level of support provided also varies. It is important to understand who will be responsible for functions like training, initial set-up and configuration, ongoing monitoring, periodic testing and recovery during a disaster.
Though cost is listed last on our list, it is probably a much higher priority for most organizations in Connecticut. Data backup and recovery is one of the budget items that is often underfunded. Afterall, what are the chances? Given the dramatic rise in cyber crime and the growing number of horror stories of the dire consequences suffered by organizations unable to recover their data, senior management is starting to recognize the business case for funding an appropriate backup and recovery solution.
There are many different backup options. They all have their pros and cons depending on which of the five variables listed previously is a priority. In all cases, a 3-2-1 backup architecture is recommended by industry professionals. There should be 3 different backups on 2 different storage media with one offsite backup location as part of the solution.
Here are the various options for offsite backup-
Employee Home — A disk or tape copy is made at the office and stored at an employee’s home. This very low-cost solution has below average effectiveness, is labor intensive, provides moderate security, and offers minimal support.
Another Office — Backups are sent or hand carried to another office. This is also a low-cost option. Depending on how the data is backed-up, it can be moderately effective and not very labor intensive. However, most offices are not very secure and lack redundant power and internet services. Security at most offices is acceptable but support may be limited depending on the solution employed.
Consumer Services — Small offices frequently employ services such as Carbonite, Backblaze, or IDrive. These are cost-effective but are typically used to backup individual PCs. They have limited ability to manage multiple servers and VMs and are not recommended for enterprise level requirements.
Public Cloud — Backing-up data to a Public Cloud such as AWS, Azure, or Google Cloud is initially very cost-effective. However, the initial “Seeding” where the complete backup, often many terabytes in size, is copied can be problematic depending on available bandwidth. It can literally take weeks to upload the original data. As an alternative, a mass data storage device may be loaded with the initial backup and shipped to the Public Cloud facility for the initial seeding. After initial seeding the amount of backup data to be transferred drops significantly. Most backup technologies employ techniques that only send the data changed since the previous backup. Public Cloud solutions are also famous for their exorbitant egress fees. A large data extraction such as would be required in a recovery exercise can be very costly. Finally, Public Cloud providers provide limited support. The responsibility for configuring a secure backup is with the customer. Public Cloud providers do not assist with testing and will not provide help during an emergency.
Local Data Center. — A local data center can provide a very effective and high security off-site data backup that is moderately priced. These facilities have professionals to assist in set-up, testing and recovery thus minimizing labor requirements and providing maximum support. Local data centers can also make the initial data seeding much easier since a large data storage device can be driven to the data center to load the backup on the data center’s storage systems. The personalized support provided by a local data center is very important when implementing a data backup solution; especially during testing and recovery operations.
CAPS has many years’ experience providing customized data backup solutions to its clients. From its secure local data center (SSAE 2018) in Shelton, CAPS’ professionals have helped clients install, configure and test a variety of data backup solutions. The CAPS’ team has supported proprietary data backup solutions in addition to assisting clients to employ advanced technologies from Datto and Veeam. CAPS also partners with its sister company, Blue Hill Data Services, to offer additional off-site data backup and recovery solutions.
Colocation, housing IT systems at third party data centers, is an effective way to increase data security, improve system availability, and save money. It also can improve interoperability, flexibility, and functionality by making available enhanced internet services.
Though most organizations do not choose colocation primarily to improve their internet service, they are frequently surprised to learn the different ways colocation enhances networking. These include-
- Increased Connectivity
- Hybrid Cloud Support
- Improved Availability
- Faster Circuit Provisioning
- Flexibility to Adjust Bandwidth
- Consistent Network Performance
- Enhanced Security
One networking benefit derives from carrier neutrality. Most colocation service providers include “Meet Me” rooms where multiple internet carriers are connected to their data center. Clients have the flexibility to choose the best internet carrier for their needs. By providing multiple internet carriers from which to choose, the colocation facility is deemed to be carrier neutral. Carrier neutrality increases connectivity options and may also lead to cost savings due to increased competition between the various internet service providers.
Connectivity to Public Cloud providers is another advantage of colocation. Organizations are building hybrid cloud solutions to make sure their workloads are hosted cost-effectively. For example, database applications can be placed at a colocation facility to avoid excessive Public Cloud egress expenses. The colocation facility enables connectivity to those applications that are cost-effective in the Public Cloud while saving a lot of money when the Public Cloud is cost prohibitive.
Colocation providers, because they have multiple internet carriers terminating at their data center, can offer automatic failover in case the primary internet service is disrupted. Border Gateway Protocol (BGP) is frequently employed to manage and automate the failover process. The ability to automatically switch to a backup internet service increases network availability.
Ordering a new internet circuit from a carrier can take up to three months. Most colocation service providers order bandwidth in bulk and proactively order additional capacity so they are ready when clients request more service. Clients can get new services up and running faster by taking advantage of readily available internet bandwidth at a colocation facility.
Organizations also benefit from being able to order the amount of bandwidth they need and adjusting up or down to respond to changing requirements. Colocation service providers can usually increase or decrease the contracted amount of bandwidth provided to individual clients in a few hours. The ability to modify bandwidth as needed to meet changing network requirements helps clients react quickly and save money.
Data centers providing colocation services often offer Dedicated Bandwidth. With the Dedicated Bandwidth option (preferred), clients do not share bandwidth with other clients. They are guaranteed contracted levels of internet service. Clients purchase internet bandwidth, typically in increments of Mbps, for a monthly fee. The dedicated bandwidth approach assures consistent internet performance.
Finally, network security is improved by colocation. Data center professionals at the colocation provider configure and monitor firewalls and security software to provide continuous protection. Physical access to the colocation facility is restricted to minimize opportunities for local breaches. Organizations that use colocation services take advantage of the data security expertise of the colocation professionals to supplement their own security team.
With the New Year, organizations are reviewing their Business Continuity and Disaster Recovery plans to make sure they are ready to respond to an emergency. There are many different service threats to consider including-
- Loss of Power
- Interrupted Internet Service
- Inability to Gain Physical Access to Offices
- Cyber Security Breaches
Though many disaster scenarios are easily contemplated, such as disruptions due to storms, floods and fires, we continue to be amazed at the variety of possible problems that can interrupt critical services. In many cases it seems unlikely anyone could predict these calamities.
For example, one client was forced to declare a disaster and relocate its essential staff to CAPS’ alternate workspace when a water main in the street near its office burst. Though the utility company quickly turned the water off, a worker was tragically killed when a manhole collapsed. The ensuing investigation required the water to remain off. Due to OSHA regulations, businesses in the area were denied access to their offices for several days.
In another case, a Connecticut bank was closed for almost two weeks due to a fire that devastated a building next to its branch office. Though the bank did not suffer any physical damage from the fire, it was forced to close by the Health Department for an extended period due to the extent of smoke that entered the branch. Fortunately, the bank had reserved Business Continuity workstations from CAPS. It was able to relocate its Call Center and was back in service about 4 hours after the disaster was declared.
If you Google the most common causes of power and data outages you might be surprised to learn that squirrels are frequently implicated. Here in Connecticut squirrels have caused many power and data outages. Squirrels like to chew on electrical wires and fiber optic cables. During the colder months they crawl into electrical transformers to try to keep warm. A few years ago the University of Connecticut was forced to cancel classes and send employees home when a squirrel was electrocuted and destroyed a transformer after bridging two high voltage lines. Every year thousands of customers lose power and data services in Connecticut due to squirrels.
Though many businesses plan on having their employees work from home in the case of an emergency, frequently the disruption that closed an office also impacts the employees’ ability to work from home. Power outages are often widespread. Homes rarely have redundant power that can sustain operations for more than a few hours. Typically, home based internet services are also not very resilient.
CAPS has been providing Business Continuity services from its secure data center in Shelton, CT for over 20 years. During this time our facility has never suffered an unplanned power outage even though major storms such as Hurricane Sandy, Tropical Storm Irene, and the October 2011 snowstorm resulted in widescale power outages throughout the state. Our alternate work site enables organizations to relocate essential operations to our facility, typically in a few hours, to minimize service disruptions due to disasters.
We provide both Dedicated and Shared workstations that are configured to replicate our clients’ applications and User Interfaces by deploying Virtual Desktop Infrastructure (VDI). Internet bandwidth with automatic failover to a secondary provider is also included. Our skilled professionals assist our clients as they perform Business Continuity tests one or more times each year to assure their backup facilities will be available when needed.
Provisioning an alternate workspace at a secure facility such as CAPS is essential to mitigate the impact of a costly service disruption whether or not the cause of a disaster can be predicted.
Hybrid Cloud Computing is one of the hottest IT topics these days. Hybrid Cloud is when multiple cloud service providers and/or cloud deployment models are used by a single organization. The hybrid cloud model seeks to optimize performance and cost savings. An example of hybrid cloud computing would be an AWS Public Cloud client that also uses Microsoft Azure or Google Cloud for some workloads. Another hybrid cloud example would be when an organization deploys a public cloud solution for some of its applications and a private cloud solution for other workloads.
By adopting a Hybrid Cloud approach, organizations can avoid vendor lock-in and preserve flexibility to select the best performing and lowest cost service solution for a particular application. Hybrid Cloud Computing should make it possible to switch services to achieve the greatest possible savings.
Yet the ability to shift quickly from one cloud service to another to benefit from price variability is not practical for most organizations. Even if an “On Demand” Cloud service contract from Cloud Service Provider A can be replaced with a more competitive contract from Provider B, there are often complications that impose barriers to such frictionless redeployments.
Amazon, Microsoft, Google and others are waging an epic battle to increase their respective IaaS market shares. They each employ legions of marketing experts who seek to get and keep new clients. The cost of transferring software licenses between cloud service providers is a prime example of a competitive mechanism that Microsoft has employed. Egress fees associated with downloading data from a particular cloud service can also make the perceived value of moving to an alternate provider far less compelling. Finally, the technical resources needed to relocate workloads from one cloud service to another can be significant and can provide an impediment to making a change.
The Public Cloud vendors have also, in many cases, made it extremely difficult to compare the costs of various IaaS services. Though they all provide online calculators to help users estimate the cost of a particular configuration, there are many variables which make it difficult to make an apples to apples comparison. AWS and Microsoft have such extensive billing options that it often requires special expertise to determine the best deal. Google Cloud is generally recognized as having the most transparent billing of the major vendors but it still is challenging to compare costs versus the other providers.
Early adopters of Hybrid Cloud Computing have found utilizing colocation services can make it easier to evolve to the Hybrid Cloud model. By moving their servers from a corporate data center to an enterprise class colocation facility they benefit from superior connectivity options. Enterprise class colocation providers offer Carrier Neutral internet connectivity. Clients can take advantage of this networking flexibility to connect to Public Cloud Service Providers. This makes it possible to benchmark various options to determine the best solution for a particular workload before committing to a service. Other colocation benefits include superior power and environmental systems redundancy and better security. The colocation provider’s technical support team often can help guide clients as they seek to optimize operations via Hybrid Cloud Computing.
A recent study by the Uptime Institute reported about 30% of all surveyed data centers experience a significant outage each year. “Data center owners and operators know that on-premises power failures continue to cause the most outages (33%), with network and connectivity issues close behind (31%).” Though power and connectivity problems often are the cause of an outage, the Uptime Institute survey indicates about 70% of data center outages are precipitated by human error.
This got us thinking. What is so different at CAPS? We have not had an unscheduled loss of power at our secure SSAE 2018 data center in over twenty years. This includes during Super Storm Sandy and several other major storms. Why is our performance so much better than the industry as a whole?
Sure, our data center includes redundant UPS and redundant diesel generators. The UPS systems provide power immediately if primary power is interrupted. Our generators spring into action 8 to 12 seconds after the UPS systems are activated and operate continuously for at least 72 hours without refueling. CAPS has established high priority status with its fuel suppliers. We have a guaranteed 4 hour delivery contract so we can run our generators indefinitely if required. We also automatically test our generators every week.
Clients also benefit from the outstanding record of service from United illuminating which provides power to the data center. UI has consistently performed well over many years through many different hurricanes and winter storms.
CAPS’ Carrier Neutral facility also provides automatic failover from primary to secondary internet carrier circuits via Border Gateway Protocol to pre-empt connectivity problems.
Though CAPS’ power, environmental, and networking systems are extremely fault tolerant, the same is true at many other enterprise class data centers. So what else is at play?
The difference may be in our people and procedures. The CAPS data center team is comprised of seasoned professionals who have worked together for years. Procedures developed and refined over the past two decades contribute to CAPS’ stellar record. Our engineers continuously monitor critical data center systems and are proactive whenever they receive notification of a possible problem. Our team all live in close proximity to our data center. Alerts are sent immediately via email and text messages to make sure the status of all vital systems is known at any time of the day or night.
CAPS also benefits from the support it receives from its sister company – Blue Hill Data Services. The professionals manning the Network Operations Center 24/7/365 at Blue Hill also monitor CAPS’ systems. This provides an additional layer of monitoring and a second pair of eyes to make sure everything is operating properly. Blue Hill also contributes the expertise of its networking and security professionals.
Finally, credit must be given to the service contractors who perform regularly scheduled preventative maintenance to insure operability of the critical equipment at the CAPS data center. CAPS has long term relationships with these providers. They understand our requirements and are very responsive in helping solve problems before they impact service. An inventory of critical spare components is stored at CAPS to facilitate and expedite repairs when necessary.
To perform well above industry norms requires a dedicated team with proper training, thoughtfully conceived procedures, and a commitment to providing outstanding service and availability. CAPS is very proud of our remarkable record over the past 20 years and is dedicated to keeping the string of success going.
Connecticut’s Department of Labor hosted a breakfast seminar entitled, “Cyber Security- What You Don’t Know Can Hurt You” on Thursday February 21st at their office in Wethersfield. The seminar was attended by professionals from the healthcare, banking, legal, construction, IT services and government industry sectors.
Capital Community College Assistant Professor of Networking and Cybersecurity, Saaid Elhadad, was the featured speaker. The three hour session flew by as Professor Elhadad offered suggestions about everything from best practices in setting up firewalls to whether you should accept a free USB drive from a trade show … you shouldn’t.
Professor Elhadad recommended implementing a layered approach to achieve the best protection. To architect the most resilient cyber security, organizations should install separate modems, firewalls, and wireless access points. Each of these devices have their own cyber security capabilities. The collective effect of this multi-layer approach is a much higher level of protection than depending on a single multi-function device.
There were many other pointers, including:
- Use two different browsers; one for general use and one for more secure financial transactions
- Wireless communications, when implemented properly, is more secure than wireline
- Use a Password Manager; Don’t rely on your browser to automatically insert passwords
- Check https://haveibeenpawned.com to see if your information is exposed on the Dark Web
- Don’t trust WiFi connections at public locations — Configure a HotSpot with your phone instead
- Implementing the Professor’s recommendations will reduce exposure by 80% but you still need proper data backup. The only protection against Ransomware is to have adequate backup
Thanks to Mark Stankiewicz and the team from the Office of Workforce Competitiveness at the Connecticut Department of Labor for putting this great seminar together. The only problem with the event was that there was not enough time to cover all the material Professor Elhadad had prepared and to answer the many questions from the audience, even after 3 hours!
The good news is Mark said they will look into scheduling a follow up session. Keep your eyes open for that as it will certainly be worth attending.
Considering whether to build or expand your data center or relocate your IT systems to a colocation facility? In almost all cases, colocation is the better choice; especially for small to medium requirements. One reason is the cost and major capital expenditures required to build out a data center – even if only for a few cabinets – and to ensure it is secure.
Here’s what’s necessary:
- Redundant Uninterruptible Power Supplies (UPS)
- Redundant fuel powered Generators
- Redundant Air Conditioning systems
- Fire Suppression systems
- 24/7 Security Systems with Video Monitoring
- Network Switching Systems, Power Distribution Units, Cabling Systems
- Monitoring and Alert Systems
- Multiple Internet carriers (for redundancy)
- Knowledgeable personnel to operate the data center
- Certifications for the data center such as SSAE-18 (SOC 1 Type 2)
There are additional considerations for companies located in Connecticut that make the colocation case even more compelling:
|Conditions In CT||Impact on Data Center Build/Colo Decision||Favors|
|Cost of living is very high||Hard to hire and retain IT infrastructure experts||Colo|
|Many financial services cos||Targeted by cyber criminals, need top security||Colo|
|Real Estate very expensive||To Build, cost is higher, ROI longer||Colo|
|Compliance Requirements||Require secure facilities with proper certifications||Colo|
Blue Hill Data Services / CAPS exhibited at BankWorld 2018, at the Mohegan Sun Convention Center. BankWorld is New England’s largest Bank and Credit Union Trade Show. Discussion topics included the latest trends in Colocation, Disaster Recovery, Business Continuity, and Private Cloud services. Sponsoring this event gave us a great opportunity to speak with our banking customers and partners, and to support the BankWorld organization. We awarded prizes to our first and second place raffle winners and even had a “Number Game” winner at our booth.
For more information contact: Tom Parnon, TParnon@CAPScenters.com, 203-925-3914