Virtualization vs. Containerization
With system virtualization, the hypervisor provides a full virtual machine to a guest: the entire OS image including the kernel is now dedicated to the virtual machine. CPU virtualization is used to provide each guest with an exclusive view of a full system environment, and these mechanisms also ensure isolation from other guests. Hypervisor-based management of virtual CPUs, memory and I/O devices is used to define resource consumption of guests.
Which one is better?
As always, it depends on your needs. If you just want to have a number of separate instances to run applications, a container environment often provides greater efficiency, both in managing the application environment, starting the application instances, and in resource consumption. Simple modification and deployment of application environments has been a design principle of container solutions like Docker and is entirely in the DevOps spirit (guess you just have been waiting for more buzzwords).
If you want to have best isolation of environments and come from a server virtualization perspective, then system virtualization may be more relevant: Noisy neighbours are much less of an issue than with containers. While many of the container folks currently focus on improving container isolation, virtual machine isolation is still superior. Coming from physical servers, virtual servers are a natural step, and an existing ecosystem around server management can often be applied to virtual servers, too.
On z systems, Linux has good scalability (to run containers), but z is the platform with an extremely efficient virtualization technology (to run virtual servers), and it is inherent in the entire system architecture. Without having measured it, combining the technologies is probably less painful on z Systems than on other platforms.
There is a third way: both.
Combining system virtualization with containers can be done in multiple ways:
1. A Virtual Machine in a Container
Docker has quite some flexibility on where containers are deployed. One option (called “execution driver”) is to use KVM images. This allows to use all the DevOps methods of Docker and combine it with best isolation available, at the cost of having to start up entire Operating System instances when starting containers — meaning startup time and footprint. Memory efficiencies can only be realized through Kernel Samepage Merging (KSM) — less effective and efficient, but it’s a start.
2. A Container in a Virtual Machine
Conversely, you can run a virtual machine and start a container inside. The VM would not be controlled by Docker, but by existing virtualization management infrastructure. Once the OS instance is up, starting a container would then be done using Docker, and no special setup has to be performed for running containers. Again, containers would have strong isolation since the next container runs in another virtual system; footprint-wise, efficiencies would only be possible through memory deduplication techniques by the hypervisor.
2b. Multiple Containers in a Virtual Machine
This is a variation of running Docker in a VM, suitable for multi-tenancy environments. Here, the assumption is that strongest isolation is only required between containers of different tenants, and straight Linux container isolation is good enough between several containers of the same tenant. Advantages are lower number of virtual machines to run, best isolation between tenants, and enjoying all the efficiencies of vanilla Docker setups.
To summarize: it depends.
Thanks for the nice article! I like it very much.
Just one think – it seems like a manipulation from my point of view. The first example “A Virtual Machine in a Container” is unreal today, it’s very close to absurd, because there are no obvious benefits. So the next two examples in the article seem much better compare to the first, and many people will believe that solution #2b is the best. But as I mentioned before it’s very close to manipulation, because…
There is another option which is the most important and problematic – containers on bare metal hardware. Only in this case the promised density is the best. And less unneeded complexity provides maximum ROI.
Sometimes multi containers in VM work really smooth. Performance degradation is less than 5%.
Please write more. I enjoyed the article.
LikeLike
Thanks for the comment, if you want more details, I can get one of our engineers to contact you. Please advise.
LikeLike
Hi Ruslan,
thanks for your feedback and glad you liked it (in general)! Did not intend to manipulate, sorry if this line left a taste. Wanted to list the options. I agree a VM under docker is a stretch, but it provides you with the isolation you are looking for. Also, one benefit is you don’t have an (IaaS) virtualization management layer involved, but simply use Docker as your tool. I’m with you though, I admit I would not prefer this model (and yes, I think 2b is best :-).
Agree density is better if you have more containers to use. Although, at some point, you’ll be CPU constraint anyway, and then virtualization overhead is the topic.
Thanks,
Utz
LikeLike
Steven, couple more thoughts just to clarify my previous comment.
There is no sense to run Docker inside VM on production. The main issues that prevent to run default Docker containers in production mode are security and poor isolation. Plus stateless design (because the lack of live and offline migration) adds even more complexity for enterprise applications during the adoption/migration process to containers architecture.
However there is a solution to run containers on bare metal hardware. We are using combination of Docker templates and Virtuozzo containers, and we run containers on top of bare metal hardware since 2012 in production mode. You can find more details at the link https://jelastic.com/docker/.
Thanks again for the article on this important topic!
LikeLike
Great comments thanks for contributing
LikeLike
Hi Ruslan,
on x86, I tend to agree, however z Systems and its customers are virtualizing by default (even second level, even in production), so this may be just a different environment.
We might agree on: using Docker in a VM in production makes as much sense as running non-Docker workload in a VM in production. As soon as you have multi-tenancy, you’d either go for a separate physical box or use another virtual server, for the reasons you mentioned (such as isolation)
Thanks for your thoughts,
Utz
PS: looking at your link, what would it take to run jelastic and Docker on z? We’re a plain Linux.
LikeLike
Utz, we need to double check it. We have pretty good relations with IBM Pure and Power teams. I believe it makes sense to analyze z system as well. I will follow up via email.
Thanks
LikeLike