On April 21, 2011 Amazon’s East Coast data center went down and brought many high profile businesses down with them. Some of the businesses that rely on Amazon to provide their infrastructure include Foursquare, Quora, Hootsuite, SCVNGR, Heroku,  and Reddit.  In addition small or mid-size businesses that relied on Amazon felt the impact as well.

So a day later we take a step back and look at the impact.  Let’s ask some questions:

First question: Will this signal the end of cloud computing?  NO!

Second question:  Will this be the last cloud based provider to experience an outage? NO!

Third question: Will this harm the migration to cloud based providers? YES!

The first two questions are easy.  The outage is not the first and will not be the last. The benefits of cloud computing to startups, small and mid-size businesses are real and this outage will not signal the end of cloud computing. But the high profile outage may harm the migration to cloud computing.

If you are a medical practice and are in the process of purchasing an EMR for your practice, yesterday’s Amazon outage gives you something to think about. Many EMR vendors give multiple options for deploying the EMR including on-site servers that are in the practice’s office and hosted servers or applications that are at the vendor’s data center or some other hosting facility. A day after Amazon’s outage you have to ask yourself; if Amazon can suffer a complete melt down of their data center what is to stop an EMR vendor from having the same experience? You may even conclude that Amazon, one of the leaders in cloud computing, have far more resources to support their data center than an EMR vendor does.  Does this make you think that the likeliness of an EMR vendor having a prolonged outage is even greater than Amazon having one? I would answer yes. And if you do answer yes, the next question you have to ask is; can you afford to be without your EMR for 24 or more hours?

I think Amazon’s outage will impact migration to cloud based computing. It will not stop cloud based computing but it is a wakeup call for businesses that are looking to use or are currently using cloud based computing. My advice remains the same as it has been in the past. Migrate non-core functions and servers to the cloud and keep core functions and servers within your network / office. There are real benefits to utilizing cloud based services but the risk is just as real. Yesterday’s outage makes that perfectly clear.

Image: Jennifer Ellison / FreeDigitalPhotos.net

My last post discussed a hybrid strategy for utilizing local and cloud based IT services.  I concluded the post by stating that I didn’t think we were ready for all cloud based IT services.  Let’s fast forward a few years and assume that businesses can run a majority of their IT services in the cloud.  Let’s assume that reliability, security and accessibility have all matured to the point that a total cloud based IT infrastructure is possible.   A key component would be that Internet access will mature to the point that it is as reliable and scalable as other utilities such as electric, natural gas, etc.  High speed Internet access would be ubiquious and reliable no matter whether you are utilizing wired or wireless connections.  Connecting to the cloud would be as reliable as turning on a light switch in a home or office.

What does a total cloud based IT infrastructure look like?  Let’s take some of the typical IT services that businesses utilize today and compare them to what would be offered by competing cloud based services.

IT Services

1. User Authentication – basic ability to log into your network and use those credentials to access other services
2. File Services – ability to access files (documents, spreadsheets, presentations, etc).  Ability to restrict access based on defined user access lists (i.e. only the marketing department can access the marketing network share)
3. Print Services – ability to print to various printers.  Queue multiple print jobs that require the same printer.
4. Email Services
5. Database Services
6. Firewall Services – protection of a network from outside access
7. Anti-virus / Anti-malware Services
8. Line of Business Applications – EMR, ERP, Accounting, etc.
9. Document Creation – ability to create documents, spreadsheets, presentations (i.e. Microsoft Office)
10. Remote Access Services – ability to gain access to other services when you are outside your network (i.e. home, traveling or at another location)

There are many other IT Services that businesses utilize but let’s just limit the conversation to these 10.

At this point I started having trouble wrapping my mind around how a total cloud based network would look like. I decided to take the approach that the network was fundamentally the same as it is today but just moved into the cloud.  I think this is the easiest approach.  Although it is an interesting exercise of trying to figure out how the network of the future would look.  A network that tied together all other cloud based services.  For more details on how it may possibly work, take a look at Dave Winers excellent post.

Let’s take our list of 10 typical IT Services and move them to the cloud

Cloud based IT Services

1. User Authentication – these services would function basically the same except that the servers you authenticate against would be running virtually in the cloud.  Amazon, Rackspace and other companies currently offer these services.  For this mind exercise we are going to assume that you can now take these validated credentials and use them to access other cloud based services.  This would be very similar to how both Google and Microsoft use a single account to access multiple services.
2. File Services – files would be stored on other cloud based services.  Access to the files would still be restricted to user access lists.
3. Print Services – ability to print to various local printers.
4. Email Services – ability to send and receive emails would be another function provided by a cloud based solution.  The solution would include anti-malware, anti-SPAM, email encryption and other services that are now usually added onto existing Email Services.
5. Database Services – SQL Server, Oracle, MySQL databases that would be hosted in the cloud.
6. Firewall Services – protecting a network from outside access will have a much more diminished role.  Local networks would no longer contain data that needed to be protected.  The role of Firewall Services would be much simpler and less complex.
7. Anti-virus / Anti-malware Services – currently these are separate services that are applied to other services such as protecting files, email, etc.  These services would be seamlessly integrated into the other cloud based services and would no longer be a separate function.  Cloud based providers would be responsible for integrating and managing these services.
8. Line of Business Applications – EMR, ERP, Accounting, etc.  Again these services would be provided in the cloud and most likely the individual vendor of the application would provide it as a cloud based service.
9. Document Creation – documents would be created using cloud based utilities such as Google Docs or Microsoft Office Web Apps.
10. Remote Access Services – the concept of Remote Access would totally shift.  EVERYTHING would now be remotely accessible from the cloud.  This would no longer be a separate service.

A typical office would now consist of just low cost workstations, laptops, tablets, thin clients and printers.  There would be no servers and no data stored locally in the office.  There would be no data to be backed up and the cloud providers would be responsible for data backups.  The IT support requirements would be minimal and the network complexity would be drastically reduced.

Companies who’s function is to implement and support the local IT services, would have a greatly diminished role.  With local IT services all moved to the cloud there will no longer be a need for a lot of  local IT support.   Although the functions that today’s IT companies now provide would still be needed.  User accounts will still need to be setup and maintained, printers setup, email accounts setup, etc.  Although these function would not require a lot of technical skill and may be able to be performed by non-technical staff.

A business that moves their IT services into the cloud would no longer have to worry about local IT support.  They would no longer be faced with the constant workstation and server upgrades, software upgrades and the monthly expense of supporting the network. All of these functions would be pushed onto the cloud based providers.  The cloud based providers would now take on these responsibilities and factor the associated expenses into the monthly fee that they charge.

All in all, a cloudy future looks pretty good.  We are not there today but we can make steps in that direction.  Some of the benefits can be realized today.  And as the cloud becomes more reliable, secure and accessible more benefits can be realize in the future.

Regular readers of this blog have heard me discuss cloud computing in the past.  I have pointed out the potential problems of cloud computing as well as discussed incredible things you can do utilizing the cloud. So it may I sound like I am all over the board regarding cloud computing.  I thought I would discuss the cloud a little more and try to clear up my stance.

Cloud computing offers businesses an opportunity to utilize computing resources and services that in the past may have been too expensive or too complex to setup.  This is especially true for the small to mid-size market (medical practices included).  I find it amazing that businesses can utilize full blown accounting systems, electronic medical records (EMR), disaster recovery services, etc. all from the cloud.  There is no local infrastructure to setup, no maintenance and support of servers and software, no capacity planning, etc.  In theory it is a dream come true for many businesses.  Usually the pricing model on cloud based services are reasonable and it provides a fixed cost and ability to accurately budget for IT services.  What’s not to like?

As long as all goes well, cloud based services are a good thing.  But if things don’t go well then the real issues with the cloud become apparent very quickly.  Single points of failure can completely stop an organization from accessing critical IT services.  A failure in communications links (i.e. T-1, FIOS, cable, etc.) could prevent access to the cloud.  Cloud based services do fail as we have seen in recent months.  These failures can leave an organization without access to critical IT services  for hours or even days. You can’t mention cloud computing without the discussion of security.  The truth is, your data is now sitting in servers and storage outside your organization.  You no longer know the individuals that have access to the data.  You don’t have control over the backups and really don’t know how or when they occur.  You data is commingled with data of other organizations. The cloud is usually publicly  accessible and you don’t have control over how the data is protected from unauthorized access.

All of the above issues are true but in reality they are no different then if you had the infrastructure, programs and data local.  Most businesses (aside from large Enterprise organizations) have many single point of failures which could produce similar problems as cloud based services.  Security in many small to mid-size organizations are usually much worse then you will find in the cloud.  Data backup and disaster recovery is usually very sketchy for many businesses.  Anyone who has faced the situation where critical data had to be restored from backup tape can attest to the level of praying required that the data on the backup tape is valid and can be restored.

So by now you may be saying to yourself that my cloud schizophrenia is clearly apparent again.  Let me lay out a framework for businesses that want to utilize cloud based services.

Start moving all non-essential services into the cloud.  If you are a medical organization your main business is treating patients. Your main IT related services are storing and retrieving patient information to assist with treatment.  Computer virus protection, Internet content filtering, email SPAM filtering, email encryption, and accounting services are not essential IT services that directly help you with your primary focus which is treating patients.  Don’t get me wrong, these are very important services but these are the type of services that are well suited for the cloud.  If you don’t have access to your cloud based accounting system but still have access to a local EMR there is minimal impact on the treatment of patients.

Utilize the cloud for services that are far to complex or expensive to implement locally.  Businesses with single offices or minimal IT resources have difficult times implementing costly and complex disaster recovery services or off-site data backup.  As I mentioned in this article, disaster recovery utilizing cloud based services can be setup for a couple hundred dollars a month.  This is far less costly then implementing redundant infrastructure in another location.  Again these are services that are important but not critical to a business’ essential services.

Keep essential services local to the organization.    Services that support the main focus of your business (EMR, manufacturing systems, etc.) should be kept local.  If you start moving non-essential services into the cloud that will make your infrastructure much more simple and easy to support.  You can then focus on ensuring that you have a stable and reliable infrastructure to run your critical IT services that support your main focus of your business.  As you start to move non-essential services into the cloud, you will be amazed at the reduction; in complexity, in the amount of servers needed and in the amount of operational support that will be needed.   This will then allow a business to focus on ensuring that there is the appropriate capacity needed for the core IT services.  That local infrastructure redundancies are in place for the core IT services.   Moving non-essential services to the cloud and keeping essential services local will allow a business to focus on ensuring that critical IT services are designed and supported properly.

By taking a hybrid approach to local and cloud based services, organizations can get the best of both worlds. Cloud based services are extremely useful and provide efficiencies and features that are difficult and costly to setup locally.  Local IT services for critical functions provide access and security. One day everything may be in the cloud and businesses may just focus on their business and have little or no thoughts of local IT services.  I don’t believe we are a this point yet.  So for now I say, keep your head (an non-essential services) in the cloud but keep your feet (an essential services) local to your business.

A recent post over at HealthcareInfoSecurity.com has an interview with Robert Wah, M.D., of Computer Sciences Corp.  Dr. Wah gives some very insightful tips on what a practice should address when looking at a hosted EHR.  Below are some key points of the interview.

Dr Wah recommends that a practice have multiple paths of connectivity to the hosted EHR datacenter.  In practical terms, you will want at least a primary Internet connection such as a T-1 and a backup Internet connection such as a Cable Modem, FIOS or DSL.

But the other thing one has to think about when looking at remotely hosting an application like this is it is important to have multiple paths to the data center so that you are not reliant on a single point of failure. Because the classic worry that people have, and certainly I had this when I was in the Department of Defense, is…we used to always talk about what happens if a backhoe digs up the cable that runs to our data center…if you have multiple pathways to the data center so you can fail over to another pathway and not lose connectivity.

Dr. Wah recommends that a practice ensure that a contract with the EHR vendor specifically address HIPAA, security and who pays to implement any new regulations.

It is important to have in the contract what is the plan when new regulations come out; whose responsibility is it to comply with those; what is the timeframe for achieving compliance; and who bears the cost of changing the system or adding new layers of security to become compliant.

Dr. Wah goes into detail about ensuring that the EHR data is backed up.

It is important to understand at the beginning…what is the normal schedule for backup, and whether that meets the requirements of your situation…. We have a client that is a major medical center at one of the Ivy League schools. Every month, we drop a tape with the latest full backup so if anything happened to the data and they were not able to get to our system, they would be able to rely on an actual backup and the gap between the time they got it and the time they needed it would be fairly short.

Dr. Wah addresses other security issues that should be considered including; how the hosted datacenter is run, ensuring that the personnel working in the datacenter are well versed with HIPPA, and knowing what the maintenance schedule is and the associated availability of the EHR.

Well I think it is important to remember that when we are talking about healthcare, in most cases we are talking about mission-critical data. So it is important to deal with it just like other industries deal with mission-critical data.

Financial industries obviously have dealt with this issue for a long time, because if they don’t have access to financial data, they are sort of out of business. Lack of access to data in healthcare can actually be detrimental to patient care, which makes it even more mission-critical than financial information.

So I think it is important to have good transparency into how a data center runs. The data center operations must be transparent to the client so that they know and have good reassurance that, as I said before, the highest level of security is being maintained both from a technology standpoint but also from a policy and procedure standpoint. The client also must be assured that the people who are working in that data center are trained, are very complaint with HIPAA guidelines, and understand the importance of electronic personal health information and are very cognizant of the mission criticality of the system that they are running.

Some people actually go visit the data center to actually see the physical plant and meet the people who are going to be involved with handling their systems. Because it is, as I said before, a mission-critical data set that they are dealing with and they want to know that they have put that in the right hands. I would say transparency is a question that you always want to bring up when you are dealing with trying to select someone to handle your mission- critical data. I think it is also important to talk about maintenance. Sometimes it is necessary to shut down the system to do maintenance….So it is important to make sure that everyone understands what the procedure would be when that maintenance occurs.

In some systems, it is possible to do it during the off hours when no patient care is going on. When I was in the Department of Defense, we had a problem where we were operating our system in 12 time zones, so there really was no “middle of the night.” Everybody was accessing the system all of the time, so we had to have backup systems put in place while maintenance was done on the main system. But other systems that are not spread as globally as we were in the Department of Defense may not have that same problem.

Knowing when the system is going to go down and when it will come back up is critical so that people know to prepare and have a contingency plan where they can go to some sort of an alternative format, whether that be paper or another system, while the maintenance is going on.

I think Dr. Wah points are very valid and give a good insight into what should be discussed with any EHR vendor that is offering a hosted product.  I have discussed some of the dangers of cloud computing in the following posts.

• The dangers of cloud computing
• More black clouds over cloud computing

In this post I talked about the dangers of cloud computing.  I mentioned that the service Entegration utilizes for it’s Help Desk support system is from Intuit and is called QuickBase.  At the time of the original post Intuit had just experienced an outage of the QuickBase service that lasted around 24 hours.

Intuit has now experiencing a second major outage that has the QuickBase service unavailable for almost two days. That’s right, QuickBase is unavailable for almost 48 hours.  Our customers are not able to log support tickets or get the status of previously submitted support tickets.  As I mentioned in my original post, we have alternative methods for clients to interact with us regarding support.

The issue here is not Entegration’s Help Desk support system, the issue is the reliability of cloud computing.  Frankly I put a lot of trust into Intuit because of their company size, the resources they have, and their reputation.  This last outage has made me think twice about utilizing the QuickBase service moving forward.  But more importantly it has made me wonder about the reliability of any cloud based service.  Entegration can survive without our Help Desk support system but what if it was an EMR?  Could a practice afford to be down for 48 hours without access to electronic medical records?  Could an accounting firm be without their accounting system for 48 hours?

Cloud computing is very attractive but cloud computing can have have a dark side.  When the application that runs in the cloud goes down, it can bring a business to it’s knees.  Most of the time there is no alternative to running a cloud based application locally so when the application is down there is nothing a business can do.

Many practices are evaluating cloud based EMRs.  I have been fairly neutral on the prospect of cloud based EMRs.  I have thought to myself that if a practice doesn’t have to invest in a lot of infrastructure and they can be up and running in a short period of time then a cloud based EMR makes a lot of sense.  But now that I have experienced first hand the dangers of cloud based computing, I would be highly skeptical of utilizing a cloud based EMR.  The dangers are much more real then I imagined in the past.

Cloud computing is all the rave right now.   It is the ability to run applications, store data and access the applications and data from any computer that is connected to the Internet.  Companies utilizing cloud computing can run applications without having to worry about servers, installing software or upgrading the applications.  Data backups are handled “in the cloud” and businesses do not have to perform daily data backups.  When you look at this model it has a lot of appeal.

A company running Google’s Gmail email service is a good example of how cloud computing works.  Google manages the server and network infrastructure that Gmail runs on.  They make changes and upgrades to the service and the users of the service see the changes without having to install new client or server software (this assumes that users of Gmail are accessing the service via a web browser).  Google adds additional server capacity without users knowing or caring about the network infrastructure.  Backups of data are handled by Google without any user intervention.  The Gmail service can be accessed by any computer, tablet,  and smartphone that has an Internet connection.

When you compare the Gmail model to the traditional Microsoft Exchange / Outlook model the differences are very apparent.  A company running Microsoft Exchange email would need to have a server(s) that runs the Microsoft Exchange software.  Every computer that accesses the Exchange server would need Microsoft Outlook installed.  The server would need to  be maintained and new security patches, service packs and software upgrades would need to be applied to the server.  Upgrades to the Outlook software would need to be installed on each of the client computers as well.  The Exchange server would need to be backed up at least nightly.  Eventually the server will become outdated and will need to be replaced with newer hardware.

The big difference between the Google Gmail model and the Microsoft Exchange model is that Google handles all of the backend administration.  The Microsoft Exchange model requires that companies, not Microsoft,  handle the backend server administration.  Companies either have to have the IT skill set or they need to outsource the function to another company.

Again you can see why the cloud computing model has its appeal.  When the cloud computing model works as it supposed to, it is very hard to choose the traditional in-house server and application model over the cloud computing model.   Companies love not having to maintain a network infrastructure or greatly simplifying their network infrastructure  and they love pushing all of those backend administration functions “into the cloud” and having someone else handle them.  There are many other aspects to consider when comparing the cloud computing model to the traditional in-house server / application model.  These aspects include security of the data, compliance issues, portability of the data to another vendor/application, etc.  For this article I will not go into those details.

Dark side of cloud computing

Let’s take a look at the other side of the equation regarding cloud computing.  Companies that rely on an application that is hosted in the cloud are helpless when the application is unavailable.  The same can be said about the tradition in-house server model but at least the company can have contingencies in place which may include additional servers, redundant infrastructure, etc.  When an application that is cloud based becomes unavailable companies that utilize the service can do nothing but sit and wait until the service is restored.

Entegration utilizes a cloud based service from Intuit call QuickBase.  QuickBase is an incredibly robust application development environment that is totally cloud based.  Intuit keeps adding additional features to the service and it keeps getting better and better.  Entegration has built a Help Desk support system using QuickBase.  All of our clients access the Help Desk application via a browser and run the application within the QuickBase cloud.  Even though we are an IT company, not having to purchase infrastructure and maintain the infrastructure was very appealing to us when we choose QuickBase.  Not having to worry about server capacity and the cost associated with adding capacity, as we grew, was also appealing.  There are probably at least 10 other reasons why going with QuickBase was the right decision but I will save those for another article.

Unfortunately yesterday Intuit suffered a major outage that affected some of their websites including the QuickBase service.  The service was down from around 10pm EST on Tuesday until around midnight on Wednesday.  This outage was almost 24 hours.  During the service outage, none of our clients could access our Help Desk system.  There was nothing we could do but sit helplessly and wait until the service was restored.  Luckily for us we have alternative methods for our clients to contact us including email and phone options.

To us, our Help Desk system is an important  application.  Our clients use the application to interface with us and we have built our workflow around the application.  But as I mentioned we have contingencies in the event the Help Desk system is unavailable.  But it gets you thinking about other cloud based services and the impact of outages.

• What if your EHR / EMR is cloud based and you couldn’t access patient information for 24 hours?
• What if your Email is cloud based and you couldn’t access it for 24 hours?
• What if your Accounting system is cloud based and you needed to print pay checks but the service was unavailable for 24 hours?

Intuit is a major technology company with many products and a sophisticated network infrastructure.  This is not a small vendor that has few customers.  If a major cloud based outage can happen to Intuit, it can happen to any vendor / company.  Intuit has the personnel and knowledge required to bring the service back online.  I would venture to guess that they will learn from this event and add additional redundancies and take steps to prevent this from occurring again.   But some questions to ask are; What if a major outage happened to a smaller vendor?  What if a smaller EHR / EMR vendor had a major outage?  Would the outage last more than 24 hours?  Would they have the resources to quickly bring the service back online?

The QuickBase outage reminds us of the dangers of cloud computing.  Cloud computing has its appeals but also has its risks.  When making a decision on an application and trying to decide whether it is best to run the application in-house or go with a cloud based solution, keep the QuickBase case in mind.