News Stay informed about the latest enterprise technology news and product updates.

Linux Kernel 2.6.23 = Win for Xen and KVM and a loss for VMware

How many of you believe that Apple’s 1.1.1 iPhone update accidentally bricked modded iPhones? Personally, I try to air on the side of optimism, but there are certainly many people out there that think Apple intentionally went after those individuals who took it upon themselves to jailbreak and unlock their shiny gadget-of-the-moment.

Here we are again, not even a month later, and the new Linux Kernel, 2.6.23 was released on 2007/10/09. The latest product of the world’s greatest hackers includes a bevy of new features, including increased support for Xen and KVM and two open source virtualization solutions. Users of those products are probably very happy today, eagerly awaiting the adoption of the new kernel by their favorite distribution in order to take advantage of the increased guest support that comes with it.

VMware Server users on the other hand are getting the proverbial shaft. Kernel 2.6.23 has one MAJOR change and one minor change that completely break VMware Server.

For purposes of dramatic effect, I will detail the minor change first. VMware Server inserts a driver module into the kernel called vmnet. It provides magical networking gnomes that help shuffle bits in and out of VMs to the wide world of webs. In one of its source files, driver.c on line 522, the vmnet driver makes a function call to “unregister_chrdev”, a function defined in the Kernel source file “fs/char_dev.c”. Prior to Kernel 2.6.23 the function “unregister_chrdev” returned an integer value; a return value that the vmnet driver keys on in order to determine whether or not to issue a warning. Kernel 2.6.23 changes the function signature of “unregister_chrdev” to return void instead of and integer. This really hoses the vmnet module source file since it expects an integer value to be returned, and thus the vmnet module will not compile when the “” script is run. Luckily there is an easy fix. It seems that the function “unregister_chrdev” has actually returned a value of “0” despite what transpires in the function as far back as 2.6.20, a Kernel that VMware Server runs fine on. Thus the easy fix is to just edit the vmnet driver.c source file and re-run the VMware Server configuration script.

That is the minor problem that the new Kernel creates.

The major problem is a bit more cumbersome, since the fix involves either redacting a change that Linus (Torvalds) has approved for the 2.6.23 Kernel or lying and declaring that the vmmon module is GPL licensed.

But I’m getting ahead of myself. Let’s start at the beginning. A memory structure called mm_struct is defined in a Linux Kernel header file “linux/sched.h”. Prior to 2.6.23 this structure included a field called “dumpable” that would determine how memory was dumped, securely or not. Kernel 2.6.23 removes this field and lets two functions defined in “fs/exec.c” take its place: set_dumpable and get_dumpable. VMware Server uses the dumpable property in its memory management module vmmon: in the file driver.c to be exact. Since the dumpable property is no longer in the 2.6.23 kernel the vmmon module will not compile.

One might think that a quick fix would be to simply edit the vmmon source file to use the new set_dumpable function. In fact, this action will result in a vmmon module that compiles; however, it will not insert into the Kernel, and an error will occur that says the module contains an unknown symbol. A quick check of dmesg reveals that the unknown symbol is indeed set_dumpable. ‘What, what, whattttt,” you say. But the set_dumpable symbol IS in the kernel. That is verifiable by peeking in /proc/kallsyms.

Heh, heh. Hold on to your seats. This is where it gets fun.

The function set_dumpable is exported in 2.6.23 with the new EXPORT_SYMBOL_GPL, meaning that only modules that are GPL licensed can use it. More can be read about this decision on the Kernel mailing list.

VMware Server’s vmmon module cannot use set_dumpable because it is not GPL licensed. There are two solutions to this problem. The first solution is to edit the Kernel source file “fs/exec.c” so that “set_dumpable” is exported with EXPORT_SYMBOL instead of EXPORT_SYMBOL_GPL and compile a custom Kernel. Then, the vmmon module source file “driver.c” still needs to be edited such that the “dumpable” property is no longer used in favor of “set_dumpable”. The second solution is to edit the vmmon module source file the same way as in the first solution, but also using the macro “MODULE_LICENSE” to indicate that the vmmon module is licensed under the GPL.

Neither solution is nice, because the first one involves maintaining a custom Kernel and custom vmmon module, and the second solution involves changing the vmmon module license without permission. A long-term solution is needed where either the Kernel developers change set_dumpable to be exported out from underneath the aegis of the GPL, or VMware could license the vmmon module under the GPL or create some type of GPL-compatible shim module that in turn calls the proprietary code in vmmon.

Perhaps most interesting of all is the timing. The same Kernel that provides extended support for Xen and KVM also breaks VMware Server. Coincidence? Like I said, I try to err on the side of optimism. How about you?

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

I find it interesting that you post a link that directly refutes your argument within your argument. Heck, your argument was refuted before it even existed. I only wish I had clicked the LKML thread before reading your article. It probably would have saved me some time.
I am not sure I know what you are talking about, but I am happy to listen to further explanation. My blog post does not really argue anything one way or the other; it just points out two developments in the Kernel and how VMware or the Kernel developers need to react. Thank you for reading the post and responding!
You could also find that vmware-any-any-update111.tar.gz, which I released on July 22nd (current one is update 113, solves both problems you mention.
I like how they took away an existing functionality that isn't GPL-wrapped and replaced it with one that is. That message from the linux-kernel mailing list mentions killer penguins with chainsaws coming after anyone who applies EXPORT_SYMBOL_GPL to existing interfaces. Well that's easy to fix! Just get rid of the existing interface and replace it with something else. Many of these people are the same ones who refuse to define a consistent kernel-module API specifically to hinder the development of binary blob kernel modules. They make noise about wanting the freedom to refactor kernel internals without having to support a legacy API*, but they also freely admit that it's as much religious as technical: they want to force the proprietary module developers to put their modules into the mainline kernel source (and yes, I've read this exact sentiment expressed on public mailing lists by Linux kernel developers, so I'm not making this up). *The technical issue is a red herring. They could do what Apple started doing in OS X 10.4: standardize the kernel API in a structured way that facilitates deprecating old APIs when the time comes, but at the same time keeping them around for a certain length of time to give module authors time to update their code. It takes more time to get rid of an old API this way, but it still works. I call this "religious-based programming": "You're not allowed to use my API unless you believe the same way I do." It seems to afflict people who care more about their forcing their views on others than about producing software that anyone can use. That l-k message you reference demonstrates it better than I ever could: > Some kernel developers are unhappy with providing external interfaces > to their code, only to see those interfaces being used by binary only > modules. They view it as their work being appropriated. Whether you > agree with that view or not is completely irrelevant, the person who > owns the copyright decides how their work can be used. I fail to see how this attitude of selfish arrogance is any different than the attitude of the movie and music industries, which seek to use ever more restrictive forms of DRM to control all aspects of how consumers make use of their content. EXPORT_SYMBOL_GPL is essentially a form of DRM: "You can only use this API in ways that I allow you to." This is all quite ironic, or perhaps hypocritical, since the religion in question claims to be all about freedom. These people particularly despise DRM when the content companies use it, but they're completely blind to the fact that they're doing the same thing themselves. I guess it's OK for them to do it because they have purer motives. And no, I don't dislike the GPL, I just dislike how some people choose to use it as a sledge hammer to bludgeon non-believers. It's people like us, who just want to Make Things Go, who end up paying the price.
Which version of VMware Server uses Kernel 2.6.23 ?. ESX server 3.0.2 uses Linux version 2.4.21-47.0.1 What is missing here.
I don't know why you are so against vmware. I find it to be a tremendous linux advocate and am proud and happy to be using it. In response to your compiler concerns, I am happily using vmware-server-1.0.4, along with the vmware-any-any-update113 patches (documented many places) on no less than 5 machines without issue (all running windows-xp).
Jim, Against VMware? Are you joking? I am very much in favor of VMware :) Are you sure you are using the 2.6.23 Kernel? It only came out a few days ago. Maybe you think I was implying that VMware Server somehow uses this Kernel. That is not what I meant. Read my response to a similar comment below. Everyone, The any-any patch did not work for me on 2.6.23. I was testing it on FC 8 Test 3 (7.92) after running "yum update" to update the Kernel. Janardhanan, I apologize for not clarifying my remarks. I am referring to a Linux distro that uses 2.6.23 that you install VMware Server on. Does that help clear things up?
Actually I am now using and my systems are debian/stable and ubuntu/feisty. no problems here. compile your own ... That said, I do wish that the vmware folks and the kernel folks would work more closely together. There really is no reason why the new stable kernel should not always work with vmware when it is released.
Jim, You are running Feisty with I would like to talk to you about the process you used to achieve this. My e-mail is akutz BLARG at FUBAR lostcreations dot NOT SPAM com.
I find the conspiracy theory unlikely. The kernel maintainers are not in the habit of deliberately breaking software that their users need to run their servers. What is far more likely is that they did not test the impact of the changes on every piece of third party software out there - including VMWare. This is not surprising. The GPL module licensing issue is likely a general change, the impact of which on VMWare was either not considered, or more likely, not even known. Expect a fix from the kernel team shortly - and until then, since this kernel just came out, it's hardly a big deal. If you're running servers, you don't arbitrarily update your kernel unless you really need the enhancements, anyway, so most users running VMWare probably won't be affected until a kernel fix is implemented. How that fix will be implemented is a good question, but I doubt it will involve any religious wars on the part of the kernel maintainers. Linus is not into religious wars, and when he sees this problem, it will get fixed.
Well, this has been going on in kernel development for some time already. Check this Linus response : Also note that apparently there was a notice sent in December of last year about GPL compliance for kernel modules. It seems that kernel developers jumped the gun and broke their own deadline. How can we forget that "collaboration" is bad word for Linux kernel hackers. And if someone dares to use Linux for other work than compiling kernel that marks that person as a worthless fool. VMware guys are not much better (except petr, :-). They could at least follow Linux Kernel development and post on their web site that new kernel breaks VMware modules on the day of release, so we wouldn't need to scavenge the web for answers. Every time I upgrade the kernel I have to go through hoops to get VMware server working again. Vmware fixed many things and worked good until I upgraded last night. I feel like I stepped on a chewing gum - stop, drop everything I was doing and spent a couple hours on something that I could have avoided if I chose different path.
"[...]There are two solutions to this problem.[...]" I think the third one might be actually copying the set_dumpable() implementation into the linux/driver.c source file.