Donations is a topic that can become a touchy one: That’s understandable because we’re talking about asking people to part with hard earned money, so it’s worth making sure that people know what it’s all about.

In the sections lower down, I’ll go into some detail of why MO:ULa needs donations and what they’re used for. You only need to read those if you want all the gory background; the next section is the “short form” for those that want a quicker read.

Donations are a “gift”

There has been some comment, even complaint, that there is a constant demand that “fans must donate.” Now, I’m not sure that I’ve ever personally seen anyone being quite as insistent as that, although I believe there is a fairly consistent level of activity around the cavern and forums to remind people about donating to MO:ULa. But we need to remember that by definition a “donation” is a gift, freely given. Later, I will explain why donations are essential to MO:ULa, but the principle must be that no-one feels coerced into donating, nor should they ever need to explain why they don’t wish to donate.

Personally, I think that things like “CAVCON Awareness Parties” or someone going round the city calling “Hey, it’s one week ’til the end of the month and CAVCON is only at 2.5 – Please consider donating if haven’t already done so” aren’t a bad thing. It’s quite easy in the hustle a bustle of daily life to forget about donating if there’s nothing to draw your attention to it. Even the CAVCON message on the MO:ULa login screen becomes a bit “invisible” after you’ve been seeing it for a few months.

So, reminders are good. Badgering people is not so good. Not everyone is able to make a donation, even if they wanted to, so you shouldn’t ever put someone in the embarassing position of needing to explain why they don’t donate. For similar reasons I get a bit uncomfortable when people talk about exactly how much or how often they donate – it places needless expectations on others. Donations are voluntary and people should only be donating what they can afford, when they can afford it.

Donation fatigue

A few months back, Marten made a very good observation to me, that is worth repeating here. Big donation drives can actually turn out to be counter productive, as you can tap your resource so heavily that you drain it dry. Someone could organise an event with the aim of reaching the elusive “CAVCON 5″ status, and it might even be successful. But at what cost? Once people donate their hearts and souls into reaching that target isn’t there a real danger that they’ll simply be unable or unwilling to donate again for some time? Possibly long enough to cause CAVCON some real harm.

I think it’s a real concern and one that I hadn’t really considered before Marten mentioned it. So strong encouragement to donate makes sense when there’s a need to recover from a month or two of sitting at CAVCON 2. Otherwise, gentle reminders to make sure that we keep just above the CAVCON 3 break even point.

CAVCON 5
The idea of CAVCON 5 is a bit troublesome for me. That might seem odd but the question I keep asking is “what would you do with it?” If we’re being honest, I think we recognise that Cyan is maybe no longer in a position to deliver any realistic new content, other than maybe the odd new t-shirt design – there simply isn’t the staff anymore, sad though that may be.

New content is going to come from the fans, and they don’t need CAVCON 5 to deliver that content. There are other things they need, such as licences, but that’s a different story.

Getting to CAVCON 5 means building up a substantial reserve fund, but for what if it’s not going to be used for new content? Sure it’s good to have something in the piggy bank for times when the donations get lean, but too much of a reserve seems to be wasteful. Or what if CAVCON 5 is just a blip (see “Donation Fatigue”, above)? You blow the reserve on a few new items then find you’ve got a shortfall in the following months and things get sticky. Personally, I’d be a bit scared to use the CAVCON 5 fund. Just in case.

So, I’d rather see CAVCON ticking over steadily in the 3 to mid-4 region rather than swinging between highs and lows as we’ve been seeing recently:

Monthly CAVCON State

Now, the boring detail bits:

The need for donations

If MO:ULa is being offered as “free” why are donations needed and what are the donations used for? Well, I partly covered this in the CAVCON article, but it’s worth expanding on this. Firstly, what we have to recognise is that running a service like MO:ULa has unavoidable operating costs that have to be met somehow, and Cyan has chosen to ask for voluntary donations to meet these costs.

But if there are these costs, can it really be considered “free?” Well, it is maybe worth noting that what Cyan is doing here is far from unique: A lot of “free” software relies on donation based funding to keep the development project alive. Many of the Linux distros operate that way – if you download Ubuntu Linux, for example, you’ll be presented with a page asking you to donate a sum based on the “value” (in $) you perceive for various features. But that’s maybe a slightly geeky example, so let’s use an example that’s more mainstream.

There can hardly be a reader of this post who hasn’t used Wikipedia at some point, and probably some people use it on a near daily basis. It describes itself as “the free encyclopedia that anyone can edit.” So how does a website like that with over 4 million articles and 18 million registered users manage to operate “for free?” The answer is “donations” and typically you’ll see a banner on Wikipedia for a few weeks each year carrying an appeal for donations. MO:ULa is in a very similar position, just on a much smaller scale.

How donations are used

We can’t be 100% sure of what the cost of running MO:ULa is; Cyan will not release specifics of how the costs are made up, and I’d guess that one reason for this that it’d probably expose something of its internal costs and that could be advantageous to a competitor. Cyan may not be running MO:ULa as a business, but it still needs to be competitive in the areas it is trading in.

However we can make some pretty safe, educated guesses at what sort of things the donation money gets used for.

Servers

This is probably the most obvious expenditure there is. Although the server code is “closed source” we’ve been able to deduce from various little snippets of information quite a bit about the server setup for MO:ULa: We know that MO:ULa uses Amazon’s EC2 (Elastic Cloud) servers and run Microsoft Windows Server as the Operating System rather than Linux as typically used by shards. For the database, which forms the basis of the “vault”, MO:ULa uses Oracle, a commercial product. In contrast, MOSS, the open source server used on the Minkata shard uses PostgreSQL. We don’t know exactly what drove those choices as it’s different from the server setup that was used for the original Uru Live, but it seems reasonable to assume it may simply have been the platform that was available for the GameTap service. Since MO:ULa was essentially just a re-incarnation of the state of MO:UL at the point it closed on GameTap it has inherited everything from that era.

Even armed with that knowledge there are several possible configurations of the EC2 servers, so we can still only guess at exactly what Cyan is using, but we can see that there is a charge per hour plus a charge for the amount of data passing in or out of the server. Since the number of hours in a month is known, that cost ought to be predictable. But the data charge is dependant on the amount and type of usage and that’s not very predictable: A new user downloading all the game files will probably use more data for that than they will in several days of normal gameplay.

Incidentally, Cyan has stated that it plans to move from “on-demand” server instances to “reserved” instances. What this means is that instead of paying for the servers on what is essentially a “pay-as-you-go” scheme, Cyan will instead commit to leasing the servers on an annual contract, with a reduction in cost of maybe 30-40%. The down-side of this, if there is one, is that Cyan is committing to paying for a year (there’s an up-front annual charge in exchange for substantially reduced hourly rates) and gambling that there will be sufficient ongoing donations through the year to meet the costs.

Maintenance and Support

All servers require maintenance and support. The good thing about out-sourcing servers through a service like Amazon’s is that you don’t have to worry about hardware failures as Amazon takes care of all that (and very efficiently too). But, beyond the provided operating system, the software you run on the servers is your problem. The MO:ULa server is really a collection of distinct elements and many people will be aware of terms like “game server” or “auth server” even if they don’t fully understand what those mean. These are things that Cyan is responsible for keeping in good order.

I guess a lot of folks are only too aware of the KI oddities that occur in MO:ULa – they’re the prime reason (or at least a symptom of the prime reason) for the server re-starts that occur every three weeks or so. And occasionally, one of the servers will crash for no apparent reason, possibly the most noticeable case is when either the auth server or backend server fails and users can’t log in, so the relevant server needs to be manually re-started. Those things need Cyan staff to get involved and their time needs to be paid for out of donations.

Now, no-one is going to reveal how much a Cyan employee gets paid, not even as an average – that’d be too much of an intrusion. But we are able to estimate how much time is spent on a re-start, partly because we can “see” how long the logins are restricted; typically it’s about an hour and a half and longer if there’s an update involved.

Why does a re-start take so long – surely it can be done in a few minutes? For
the Build 912 update on May 29 this year, Christian Walther and I were invited to assist the support team with the update verification. And this is probably the key thing: After the re-start, the support team, Tor’i, Dexter and ResEng M.Dogherra, run through a series of checks “in-game” to ensure that all the server elements are operating and that everything is working as expected before allowing the public back in. Some of these are things that some players might not even be particularly aware of such as the “bad word” filter on KI chat but it all takes time. I expect it’s all part of Cyan’s “quality control” that they’ve used for Magiquest Online and during GameTap MO:UL, so it’s second nature to work that way. Once you throw in some of Chogon’s time youre looking at maybe 5 man-hours of effort for a re-start.

By the way, during that 912 update Christian and I concentrated on checking that the new features had made it in OK while the Cyan folks did their routine checks: By parallelling those tasks we kept the downtime to a minimum but it was still getting close to three hours. So updates will certainly cost more than a regular re-start.

RAWA created CAVCON (CAVern CONdition) shortly after MO:ULagain opened in February 2010 in response to requests from the fans for information on how well it was performing with regard to collecting donations.  Now, here’s where one of the important things about MO:ULa comes is and creates “difficulties” for giving a simple status:  MO:ULa is provided as “free-to-play” and donation is entirely voluntary – no-one is obliged to make a donation, no matter how much or how little they play.  Cyan simply asks that people donate what they can reasonably afford and there was never any suggestion that there was a “recommended donation”.  So, at one end of the spectrum there might be a few people with a substantial monthly disposable income who feel that donating $100 per month is quite comfortable and at the other end some who really struggle to make the domestic budget cover the essentials of living and couldn’t really afford to donate at all.  And a lot of people somewhere in between.

So, Cyan needed to find a way to report how healthy MO:ULa was without placing any undue pressure to donate on folks who simply couldn’t afford it, or who were otherwise reluctant to donate.  Expressing thing in terms of the cash value in the fund would lead to people trying to work out what that meant per player and concluding “Hey, we need to donate $5 each per month to keep this going” and at that point it becomes more like being charged a fee than a voluntary donation.

The CAVCON concept has a precedent: Many of the “old guard” of Uru Live players will recall the “Cautious Optimism” (C.O.) meter that began being used when the GameTap deal was being worked out to indicate the likelihood of Uru Live returning. Both the C.O. Meter and CAVCON share a root in the DefCon (Defense Readiness Condition) indicator used by the Armed Forces (and much loved by movie makers, e.g. in “War Games”).

The C.O. Meter scale

CAVCON Level Definitions
CAVCON 5: Unlike some movies, which get the DEFCON levels backwards, CAVCON 5 is the best state. It means that donations are exceeding MO:ULagain expenses, there are a couple of months of MO:ULagain expenses in reserve, and extra money can be put toward server upgrades, bug fixes, new content, incorporating fan created content, etc.
CAVCON 4: This level means that donations are exceeding MO:ULagain expenses, and we are building a small reserve fund for future slow periods.
CAVCON 3: Donations are roughly equal to MO:ULagain expenses.
CAVCON 2: Donations are less than MO:ULagain expenses, and we’re currently using the reserve fund to help pay for MO:ULagain expenses.
CAVCON 1: Donations are less than MO:ULagain expenses, and the reserve fund is empty. This would be A Bad Thing ™.

So, put simply CAVCON is an approximate measure of how donations compare to expenses. But it also implies some other things: To be meaningful, you need to know whether CAVCON is reflecting the position for the current month or over the longer term. I always interpreted it as being related to the current month and I think this was correct. However that left some other issues, for example CAVCON 5 implied that CAVCON 4 had existed for some time, and there was now way tell by how much CAVCON 4 was topping up the reserve or how badly CAVCON 2 was eating into the reserve. A number of people called for more information, seeing CAVCON as inadequate in various ways, with two points in particular recurring:

1. A scale of five points, especially since only levels 2, 3 and 4 seemed to be relevant, didn’t give any indication of how far short of or in excess of expenses the month’s donations had been.
2. There was nothing to show the overall “health” of the reserve fund.

For October 2012, RAWA introduced some changes, one of which attempted to address the first of those points by adding an additional decimal to the CAVCON value. This in itself created some further confusion, since for level 3 (donations near equal to expenses) a decimal part would be meaningless, so RAWA posted a clarification on October 14 and gave an example of the revised scale:

4.9 (Donations = expenses + 90%)
.
.
.
4.1 (Donations = expenses +10%)
3.0 <--- monthly minimum goal (Donations = 100% of expenses)
1.9 or 2.9 (in both cases, Donations = 90% of expenses, the whole number portion just tells who covered the difference)
1.8 or 2.8
.
.
.
1.1 or 2.1
1.0 or 2.0

For values of 1.x Cyan pays the difference while 2.x means that the difference is paid out of the reserve fund.

Speaking personally, I was actually quite content with the original five point scale despite its shortcomings, and feel that with this new scale RAWA has created something that is not very intuitive, but as JW Platt commented to me “He’s a puzzle man.” And I guess that as Uru players we all ought to be able to cope with a small puzzle in reading the MO:ULa status.

The other change that RAWA brought in was the Donation Meter. This is a measure of how donations are progressing for the current month. Reset to 2.0 at the beginning of each month, it is generally updated on a weekly basis and progresses towards 3.0 when the donations have met the expected costs for the month. If the donations exceed the target then the Donation Meter will move to 4.1 and beyond. Quite what happens if monthly donations exceed 190% of expenses and we need to go past 4.9 isn’t clear!

CAVCON and the Donation Meter are two distinct measures; CAVCON tells us what the final position was at the end of the preceding moth, while the Donation Meter tells us how we’re doing as we progress through the current month. However, as both get reported together, they are often both referred to as “CAVCON” leading to some folks wondering “Why did CAVCON take a sudden dive from 4.7 to 2.0? What happened?” I expect in time we’ll get used to differentiating between the two.

So what are the expenses anyway?
There are a number of questions that come up about the nature of the expenses for MO:ULa so I’ll try to address some of the more recurring ones, at least so far as I’m able.

Why should I donate to MO:ULa, especially since there’s been no new content for two years?
No-one is obliged to donate to MOULa – that’s the long and short of it. But if someone doesn’t like what MOULa is doing (or not doing) and sees “some other shards” as better alternatives then presumably they don’t use MOULa either. If they do still use MOULa, then there must be some “value” to it, in some form, so it must be worth donating “something” for that value. Provided, of course, that they feel able to donate in the first place. I think that Cyan created MO:ULa for “its fans” and not just for “the fans who are able to pay for it.” Encouraging or reminding the community at large to donate is no bad thing, but badgering individuals over it is something to be avoided.

What does the donation money get used for?
Fundamentally, two things: The leasing of the servers and “hiring” Cyan staff to support MO:ULa. Contrary to what some people assume, Cyan do not own the servers they use, and the MO:ULa servers are leased from Amazon and incur a fee for each server instance used and another fee for the volume of data passing through the servers. In addition, MO:ULa has no “staff” so it needs to pay for Cyan employees to take time out of the working day for support work, like server re-starts or updates – it is essential that MO:ULa is seen to be self-sufficient to maintain the separation from Cyan’s business.

How much does MO:ULa cost to run?
We simply don’t know, it’s not information that Cyan is prepared to share for the reasons given earlier in this post. There have been estimates posted elsewhere, derived from the published Amazon EC2 pricing and guesses at the labour hours to support MO:ULa, but they are only estimates and based on a lot of assumptions that may or may not be accurate.

Why does Cyan have these expenses when shards are free?
No shard is truly “free”, but in most cases the shard operator has decided that they’re able to bear the running costs without asking for money from the users, although I have seen at least one shard that invites donations. The shard’s server costs may well be a lot lower that Cyan’s and the labour to run and maintain the server is effectively free. Even a home PC acting as server will have some “cost” associated with it even if it’s just the usage of a portion of the internet bandwidth.

Can’t Cyan use cheaper servers?
Possibly. The choice of server setup is probably a legacy from the work done for Magiquest Online and its beta testing, since work on MO:ULa began right after the MQO beta closed. It’s possible they could move to an alternative, lower cost server setup, but the move would take time and effort (= money) so it starts to get questionable whether it’d be a good idea or not. There are probably a lot variables related to the server setup that we can still only guess at, since the MOULa server is not the same as either MOSS or DIRTsand. That said, Cyan have announced that the intend to move to an annual contract for the MO:ULa servers in order to reduce costs, saving maybe 30-40%. While this removes some of the flexibility of the current arrangement, I expect that the usage metrics are showing that flexibility is an unnecessary luxury.

As a final thought, I’ve seen a couple of comments to the effect that “Open Source means that Uru can’t really die now – the fans will keep it going.” Technically, I guess that’s true, but I’m of the opinion that even Open Source needs MO:ULa: At least for the present, if MO:ULa shut down then we’d lose all (legal) access to the Cyan ages as there’d be no legitimate location to load them from. Without MO:ULa to act as a “base” or “hub” it would seem likely that shards would become increasingly insular communities that would mostly wither and die in time, and although it’s not a driver for all potential content creators, I think that the absence of a Cyan shard would remove some of the incentive for people to work on new ages or even learn age writing. There’s a symbiosis here: MO:ULa needs fan support, both in terms of financial donations and the donation of effort from fan developers and writers, in order to survive, and Open Source Uru needs MO:ULa for it to have a focus and reason.