(RSS Feed|Atom Feed)

Virtual Networking ( By dho Last mod: Dec 29 2008 EDT )

Virtual Networking

These days, a machine is not generally useful without at least some connectivity to a public network. Indeed, the purchasing criteria for even the most basic systems (from less technically inclined crowd) has jumped from a singular requirement for word processing to including email access, web access (frequently including multimedia content), and more. All of this is due to the boom and promotion of the largest public network: the Internet.

It should then come as no surprise that, when emulating a machine, these requirements remain unchanged. Although the virtual machine in question is one specifically focusing on Plan 9, and such a machine would be largely useless without networking support, the principle is the same. The issue becomes how one provides networking support in a cross-platform manner.

Socket APIs

The easiest method of providing access to the largest public network is by interfacing with the operating system's public API meant for such things: the socket API. At least in Unix-like systems, this API has existed for decades, and is fairly well defined. The advantage to interfacing with this API is that one is able to quickly create bindings for a virtual network driver.

The major disadvantage to this approach is that one loses the ability to choose protocols. The socket library only provides an interface to networking protocols supported by the underlying operating system. While this may be sufficient access for some environments, others (notably Plan 9) can make use of protocols built on top of the Ethernet level. This is simply not possible using sockets.

Another disadvantage is that one must share the host's protocol-specific networking stack. There are many reasons this is undesirable, especially for a Plan 9 environment:

Thus, while this implementation may be straightforward and seem reasonable for many cases, it is quite arguable that it is not the ideal solution for most cases.

Virtual Network Drivers

The alternative to creating a wrapper around the system's user stack interfaces is to create a virtual network driver. This involves creating a synthetic device that runs in user-mode and is able to function to the hosted environment as some sort of stripped down network device. There are surprisingly many different ways to do this, though not all are terribly portable. In fact, there are three separate virtual network drivers in my 9vx repository, each with a different approach.

The major problem with this approach is that all descriptors must be created as the root user, since they all require some special access -- either to open a raw socket, bpf device, or to create a tap device.

Libpcap

The libpcap-based network driver was the first driver I introduced, and it was the first to run. I chose libpcap because it is portable, readily available across several operating systems, including the BSD family, Apple's Darwin, and many distributions of Linux. Additionally, the API is fairly simple. The major downside to this implementation is that it requires a bit of preconfiguration: one must determine a system interface on which to listen, as well as assign a MAC address to the interface. Libpcap also makes some silly assumptions about when it should return packets to the caller -- its pcapopenlive(3) interface has a parameter to_ms which is a timeout value in milliseconds to queue up packets before returning them to the caller. Finally, the current implementation only supports a single interface. While this could be expanded, it's not necessarily ideal.

Raw Sockets

A raw sockets approach seems straightforward; however, this method only really works under Linux. The BSD operating systems do not up-shoot headers from TCP or UDP packets to raw sockets. This makes the raw socket solution not very portable, though it is theoretically faster than the Libpcap solution, as there is no queueing involved. It's only slightly easier to add more interfaces in this implementation -- multiple filters needn't be created.

Tap Driver

It turns out that a pseudo-device has been created and implemented across multiple systems. From the FreeBSD tap(4) man page:

The tap interface is a software loopback mechanism that can be loosely described as the network interface analog of the pty(4), that is, tap does for network interfaces what the pty driver does for terminals.

The tap driver, like the pty driver, provides two interfaces: an inter- face like the usual facility it is simulating (an Ethernet network inter- face in the case of tap, or a terminal for pty), and a character-special device ``control'' interface.

Once opened, we simply attach our stack via read(2)/write(2) routines -- we can rather easily get the device's MAC address for use within our device. Since this is a portable interface with hooks at the kernel level, it's also the fastest interface we have to access network functionality using our own network stacks.

How They Work

All these drivers work in essentially the same way. The driver may initialize as it is normally expected to. When it attaches, a kproc is set up that polls a given resource for data to read from the network. If data is available to read, that data is consumed and put in the ether input queue.

Transmit functionality works as normal -- when anything pops up on the transmit queue, flush it out via the consumed API's transmit method and one is good to go.

--dho

Setting Up ( By dho Last mod: Dec 19 2008 EDT )

Community

You might say I'm a bit bi-polar with computers. I'm extremely opinionated (and elitist) about what's correct and incorrect (sometimes rather less than others, but still...) and I become interested in phases. A perfect example of this is my yearly foray into Plan 9. Each year, I find myself wondering what is going on with the Plan 9 operating system and its little community. Infrequently do we find new users; the frequent posters on the mailing list have been the same ten or fifteen people who were there when my interest was first piqued about six years ago.

The small community size is both captivating and frustrating. Those who stay in the community for any length of time are generally rather smart, constructive people. Several members of the community don't rub well with the others, and change happens very slowly, which is excruciatingly frustrating at times. Additionally, we see so many people come and go that we tend to treat all new users with some bitterness. I suppose many communities are like this: my experience in BSD communities has also shown some levels of frustration with new users, but the frustrating in this community is different. We shall always grow tired of answering the same uninformed, UN-researched questions, but our bitterness is due to the fact that new users will likely not stick around.

It's easy to place the lack of user influx on the bitterness and frustration emanated by the community, but it would be the wrong place. Certainly it has some bearing, but the bitterness and frustration are a produce of stagnation rather than a reason for it. Users come to Plan 9 because of its roots: designed by the same people who made UNIX, an operating system that has inspired several of the ones they use. They expect familiarity, portability, hardware support. Unfortunately, these things are the antithesis of Plan 9's goals. Indeed, there are myriad reasons for Plan 9's somewhat stagnant user base:

My cyclical interest in Plan 9 is due in part to building frustration with uninterested influx and in part to building frustration with internal development of the project. To me, the ideals of Plan 9 are correct; set towards the future (funny for an operating system that is over twenty years old) and ideal for development -- a plethora of accessible APIs exist for networking, interprocess communication, threading, and interface design. The small nature of the community, however, does not warrant or merit much in the way of development. Those using the system frequently are able to add features as they need and thus do not rely on others to develop for them. This makes it hard for a hobbyist to develop: when few people are using a product that is viewed as mostly feature-complete, what does the hobbyist do outside of porting to more hardware? (More hardware has been mostly inaccessible to me.)

I looked into Plan 9 again this year on a whim. The developments are extremely interesting to me: user-space Plan 9 kernels run with 9vx. They're pretty bare though; not much in the way of features. Lots of room to expand. Lots of cool things to do with them.

So I'm planning on sticking around for a while. What exactly my plan is -- well I'll touch on that a little later in this entry.

The Process

There's always a large process involved in getting interested in Plan 9. The first barrier to entry is always finding that extra piece of hardware on which to run Plan 9. This time, I had no extra hardware -- I didn't particularly want to use the publically available cpu- and file-servers -- they've always been slow and... well, they're just not mine. Then I found Russ Cox's work on 9vx, a Plan 9 kernel that runs in user space across a wide variety of operating systems. And it just works. I had to figure out how to use mercurial to a small degree, but that wasn't so bad.

After that, I needed a place to put everything. I have a public server on which I can run 9vx -- but I have a ton of websites on there already. So I picked up a VPS; its use is manifold: to provide news on my development work, to host source repositories, to play with werc, and to serve as a control node in a beginning grid.

Getting everything set up has been interesting. The VPS service is rather good in my opinion. Their interface is pretty useful, and I've gone with Ubuntu since they don't appear to offer NetBSD. Ubuntu was the smallest base install with recent packages (Debian 4.0 shit is still ridiculously old. Come on, guys. Get with the program.)

werc is pretty nifty. I'm still fiddling with it and coming up with patches here and there. It is, indeed, delightfully anti-web -- the CMS I've been considering writing for years and the one I've wanted to use for even longer. Its file-based, plain-text nature tickles me just the right way. It was also pretty straightforward to install: simply clone the repo, put 00-werc.conf in /etc/lighttpd/conf-active, and here I am being Bloggy McBloggerson.

The Plan

I have several plans for improvement on 9vx -- some are ongoing projects; some are completed:

There's always lots to do in the areas of virtualization -- any suggestions or requests will be happily queued :)

--dho