Latest entries are shown here. For the full blog, visit http://sirgolan.livejournal.com.

New Release and Moar Content on Sat, 19 Jun 2010 03:21:41 GMT

MV3D 0.42 released today! Check out the release notes or download the MV3D Windows Client! In this release, the core systems were made ready for tool building and content creation. This included a redesigned persistence mechanism and a cross system GUI layout.

The Good:

• New persistence system is pretty awesome.
  
• For the first time ever, you can export or import sections of the game world!
  
• Pretty cool start to a new way to build UIs which should save tons of time and allow for quicker iteration as well as reusability.
  
• Made an unreleased play in browser app for MV3D.
  

The Bad:

• New persistence system may have some performance issues.
  
• Tons of bugs still need to be squashed.
  
• Didn't get to any tools which was the original focus of the release.
  
• No visual features added-- nothing cool to show off really.
  

What's next? Well, the next release will focus on the content pipeline. This means upgrading Ogre (since it's long overdue) and tools, tools, and more tools. The one major unknown is how the content pipeline will actually work. Ogre has a lot of half functional or very basic exporters for various 3D modeling packages, which is an unfortunate start to the pipeline. That's not so much something that MV3D can or should solve, but it's fairly disappointing.

After exporting the raw files, they have to be added as assets in MV3D. Currently, RED can do this in a very basic way. It does include the ability to SFTP the asset files to the server that will be distributing them which is handy. This will need to be extended and made smarter and easier to use. RED should definitely get in app previewing of assets. It'd also be nice to add a preview image to assets in general. This could be displayed in RED, in the IGE, or on the web editor. Speaking of the web editor, it's completely busted more or less. That'll need fixing, which may come in the way of guide (the GUI framework which also supports web GUIs). What's awesome about fixing it with guide is that the same GUI can be used in IGE or RED without any modifications.

With the assets imported, the real fun begins. RED will need a plugin that lets you add and arrange objects in a vacuum (i.e. not anywhere live in the game). Then you can use Freeze (the importer/exporter built in 0.42) to save off pieces of the game world. An additional RED plugin should be able to connect to a server and edit existing areas. Here you could import the stuff you created before and place it in game. This should also have the same functionality of the other plugin so that you can edit anything you want.

One other missing piece is the collision volumes. These need to be manually placed (or generated from static geometry in some cases). This means the other required tool will be something which is able to position colliders for models.

But wait, there's more! There needs to be a way to edit bodies of water, and in fact, the current ocean object should probably be made into a water volume since right now it's just an infinite plane. Similar to water, the sky should be editable, and if you are editing the sky, physics properties of the realm need some love too.

Finally, it'd be nice to be able to have more than one area in the game at some point. Part of the RED area editor should include connecting areas together. At this point, it's been so long since this was tested, that it may not even work any more.

Sounds like I have quite a few tools to write. Any volunteers to help?

Storage Space on Sun, 31 Jan 2010 20:26:50 GMT

I've been trying to come up with a better way to persist MV3D state info. Basically, I need a solution that meets the following:

Synchronous when saving data

Transactional

Extremely Fast when saving data

Maintains data consistency

Supports queries

Externally available (i.e. you can view/modify/query the data outside of MV3D)

Low CPU overhead

The current solution does support most of those. The four it's probably worst about are queries, data consistency, CPU overhead, and external availability. It's also just kind of wacky. You define a list of properties in your class that should be stored along with which of those properties you'd like to generate a search index for. When it's time to save an object, it generates a uuid for it, then it stuffs all the properties you defined into a special object, pickles it, and then uses Axiom to persist it to sqlite. It uses index objects stored in Axiom which match the uuid with the value of the property in the object. Querying is also a little odd since you use magical properties on a query object (q.a == 12). Unfortunately, the object type you are querying for may not have an a attribute since the query object doesn't really care. Another interesting part of the system is that it requires all servers to store their data locally. I'm not sure how I feel about this since some MV3D data really isn't useful to other servers-- at least as the design stands now. However, it does require that there be two servers for every item stored in order to recover from a catastrophic failure of a single server.

I've tried in the past to go with a completely Axiom based solution, but that didn't work well because of the restrictions Axiom puts on any classes you mark up as Items that can be persisted. One thing I'd been thinking about was a dual object approach where you have the in memory object, and when it needs to persist, it stores all its persistable attributes into a specific Axiom class. This system could even use powerUps. This had a couple of downsides, one of which being that you had to define twice as many classes. It also ends up that you'd have to define your datastructure twice as well. The other option is to have a utility to generate that extra code, but I'm not a big fan of code generation, so I'd like to avoid that.

What I'm currently thinking of is adding a new service type to MV3D that would generally act like a data access layer (DAL). You'd mark up classes in Python with which attributes should be stored-- and I'm thinking of combining this with the attributes that are sent over the network. In order to persist an object that had this mark up, you'd hand it over to this service. It'd then store the marked up attributes and return a unique ID you can use to retrieve the object later. While technically at this point, it really doesn't matter where or how the data is stored, I'm going to go into that a bit. There would be several low level functions: register schema, upgrade schema, add object, update object, get object, and query. Basically, schema in this sense is the markup of a particular class which defines attributes to store and what type they are. For SQL based stores, this will be directly associated with a table. Schemas will be versioned so that when a newer version is registered, all data is upgraded. I see this happening by renaming the existing table, creating a new one with the new schema, and running code on each element to upgrade it.

After the schema is created, adding an object would just be a SQL insert and updating would be similar. Both of those could return the primary key of the row as an ID. Adding the primary key to the object's class and the service's location would give you a method of retrieving that object from anywhere. Querying for the user could make use of the class markup objects to give them the ability to do something like: store.query(Person, Person.name == "mike").

Looking at the requirements I mentioned, the major ones that it fails are being synchronous and maintaining data consistency. I put synchronous when saving up there because I like to be able to wrap methods that modify data in a @autoStore decorator which stores the object after the method runs. If this were async, either that function would have to return a deferred, or you'd lose data consistency if there was a failure. It would also be sweet to be able to define the markup classes as descriptors which would a) store the object and b) update any network clients of the changes. A possible solution to this would be to strategically limit the remotely available functionality of the service to the low level operations I mentioned previously. Then to make them have nothing to do with converting objects into persistable data. This would require a local object/service to interact with the remote store. The local code could build a queue of transactions to send to the master store, and this queue could be persisted via sqlite. This way, the data would be synchronously stored to disk locally and then sent off to the remote store whenever.

One other issue here is that theoretically multiple servers could access the same object in the store at the same time. Yes, that sounds like a feature, but the aforementioned queue causes some issues with doing this. The data in the master store may not be 100% up to date. What might be interesting is to create a locking mechanism whereby when loading an object from the store, you acquire a lock on it. The hard part will be making sure that the lock goes away whenever you stop using the object and that failure conditions (such as your server crashing while holding locks on multiple objects) are properly handled.

Another issue is what to do if the store rejects a change that seemed like it would succeed locally. If we go with writes that return immediately and don't wait for success, it would already be too late to tell the original caller that something went wrong. What are some of the reasons this would happen?

Remote storage server is down. Ok, just keep it in the queue until it's back up.

Remote storage server is out of space. Keep it in the queue is probably best.

Schema mismatch or invalid data. This seems like the worst failure mode.

What do we do with a schema mismatch or invalid data? This indicates a fairly serious problem, so maybe it deserves a serious resolution-- revert the object and all other objects related to the transaction that failed both on the remote store and in memory. That still doesn't seem like a great way to resolve the issue, but it would maintain data consistency.

All in all, this seems like it'd be pretty challenging to implement, and it'd mean that I'd be maintaining a DAL/ORM as opposed to using a pre-existing one. I'm generally against re-inventing the wheel like that, but all the Python ORMs I know about are designed for webapps and don't translate well to MMOGs.

Really at this point, I'm looking for someone to talk me out of this and tell me why it's a horrible idea. Otherwise, I might just be crazy enough to try it out. One thing that's bugging me is that the current mechanism works. I haven't had any lost data issues or anything; however, I've found that sometimes it's just easier to blow away the whole store than to say revert a change I made.

At the very least, I like how the new system abstracts out the DAL to a service that can optionally be on a remote server. This makes it so the underlying persistence technology can be pretty much anything. Another major benefit would be the ability to freeze objects by storing them and stopping simulation of them. Then you could unfreeze them later on a different server. What's this good for? Well, making it so your character leaves the world when you log out for one. This isn't currently possible without a load of hacks.

Ship it! on Sun, 24 Jan 2010 20:10:27 GMT

MV3D Version 0.40 will be officially released today. I'm building the Windows binaries and source release now. I'm happy with the release, but visually, it's a bit of a step backwards. I don't have any good screenshots to show, or a movie detailing the new features. There are new features though. They just aren't visible, and if they are working properly, you should never need to worry about them.

The main theme of this release was completely refactoring the scalability and redundancy of MV3D at the core. This was a very big success. I now feel comfortable saying that MV3D is ready to scale. In a properly configured cluster, items will be both spread across the servers and have redundancy in the case of a failure. Items can be transferred from server to server based load balancing needs.

Looking at MV3D as a whole, there are still a few elements that need major work. The most obvious right now are tools, the client, and possibly data storage (sadly). It seems to me that tools to generate content are less useful if the data storage mechanism changes significantly (since either I'll have to write a converter or you'll have to lose all your stuff). On top of that, making the client look good and perform well is hard to do and mostly pointless without any content.

With that said, the logical order is to work on data storage first, then tools, and finally fix up the client. I'm hoping to not spend too much time on data storage. I have some ideas, and generally want to make whatever I come up with retain the same API. For tools, RED needs to be extended and generally made useful. There also needs to be a way to more or less export and import whole areas (or even whole realms) at a time. Any project bigger than a single person messing around will want to have this ability so that they can version control their world.

While fixing a recent bug, I came to realize that a tool for managing MV3D clusters is desperately needed. My repro steps for that bug involved checking in to SVN, updating two servers, restarting them, and then connecting to their SSH console. It would have been nice to be able to run 2 or more servers on my desktop for testing. So, this is another tool that should be forthcoming.

While I'm talking about that, the reason I had to repro the bug on the servers as opposed to in unit tests is because while there are unit test facilities for testing a full server against a real-ish client, there isn't much in the way of full integration tests between multiple servers. Some of the problems I was tracking down ended up being issues that came up only when you started two servers, created some stuff, shut them down, restarted them, and tried to create more stuff. Integration test helpers for scenarios like this would be pretty awesome.

Another headache I ran into while rebuilding the demo servers was that since the starter world initialization code is written to work on a single server that provides all services, it doesn't work well when you have a separate directory or account server. This made it so that I was basically typing the code on the console to create the world. No one should be expected to have to do that-- a tool is needed that can bootstrap a set of servers to get them to the point where other tools like RED can be used to create a world.

There are plenty more things that need tools, but so I don't bore people more than usual, I'll move on to the client. The client seems slow. In reality, it gets 150-160 FPS, but it still feels sluggish for some reason. This is probably physics related, but it makes things look bad. The other big issue on the client is that since the server sends position corrections to the client's avatar instead of the other way around, moving around can be a little clunky (even with the smoothing that's done now if there's a discrepancy). The UI on the client is very clunky and ugly. I'd really like to get that looking better. There's tons of stuff really that needs help.

Sounds like a lot of work, doesn't it? It definitely is. If you've gotten this far, you must be interested in MV3D, so why not contribute? There are a ton of open tickets now that would be excellent for someone looking to get started.

That's all for now! If you try out the new release, just be warned that it's visually the same as the last release. I'm hoping that as tools come online, I'll be able to add more content to the demo world, but that depends on what code changes I have to make on the server to support the tools.

No news is.. Well, no news. on Mon, 14 Dec 2009 03:02:35 GMT

It's been quite some time since I've posted anything here about MV3D. Things have definitely been going slowly recently in that regard, though they are picking up again now. I work on an MMO all day, so when I get home, I generally want to do something else for a while. However, now that I'm more settled in, I've been able to start putting in the time again after work and on weekends.

Mostly, I've been trying to put together some content editing tools since the in game editor isn't really the place where you'll want to put all your content together. It's good for terrain sculpting and adding foliage and such, but I think I'd pull my hair out trying to use it to make a town. One of the major methods of building complex things in MV3D is the modular object system (demo'd in a few of the videos I think). I'm working on a tool so that complex modular object systems can be designed without having to write code. It's coming along really well. At this point, I'm not sure it really fits in with MV3D, so I've been keeping it on the side.

My goal is to get content creation and some semblance of an art pipeline up to a point where I can create a passable looking game world. I'm not expecting anything too wonderful because I have no talent making 3D models (and am therefore limited to GPL or CC licensed ones that I can find), but I should be able to come up with something. Then after that, I'm going to start working on putting together a demo game.

At this point in the development of MV3D, I've often been unsure what the next most important thing for me to work on is. If I start putting a game together, I'll be able to quickly see what areas need help and work on those. At least that's the theory.

One thing I really want to fix up is the client performance. Last time I ran a load test, having 50-100 characters running around caused the client to become fairly unresponsive. Not surprising since I'm not doing a ton to mitigate that.

I'm sure some people are probably hoping I'll talk about 38 Studios and Copernicus. About all I can say is "Yes, it's awesome, and no, I can't talk about it." Ok, I can say more than that. We're hiring! Check out the 38 Studios Jobs Page. It's really a ton of fun working with such an incredible group of people.

Hopefully, with things picking up on MV3D again, I'll be posting a bit more. I've also started an MV3D page on Facebook. Mostly, I'll use that for random related things that don't really warrant a whole blog post. I'll probably also put pictures up there that I wouldn't want to put on the main page.

Lack of Windows 64bit support on Fri, 15 May 2009 03:48:51 GMT

Seriously. I have XP 64 on my main system at home (ok, stop laughing) because I have 4gb of RAM and don't want to make my Athlon X2 into a K6 by running Vista. Anyway, I've recently decided that it would be a good idea to move my music creation stuff from my old desktop to the new one. The old one just really can't keep up any more and starts getting tons of lag when doing real time effects. Anyway:

M-Audio Midisport 1x1: no drivers for XP 64 (most M-Audio hardware: no drivers for 64 bit OSes)

Every MIDI loopback driver except LoopBe and MOLCp3: no support for 64bit OSes

Behringer FCA-202: no 64 bit drivers period (most Behringer hardware: no drivers for 64 bit OSes)

Cakewalk 64bit version: runs ok but seems to not detect some VST plugins (32 bit ones maybe?)

Cakewalk 32bit version: BSOD.

The last one is the one that really irks me. Apparently, the reason it blue screens is because Microsoft never made ASIO sound drivers work properly in XP 64 or Vista 64. Seriously? ASIO, by the way, is pretty much a requirement if you want minimal sound lag.

Apologies for the lack of updates lately. I've been plugging away with refactoring load balancing and am getting close to done. I'm considering another release shortly after that stuff is finished. I'll probably put in some things I was hoping to get in to the last release, along with some bugfixes and then call it done. Other things have been keeping me busy such as my new job, which is completely awesome of course!

LFG

Balancing act.

Redundancy, check.

Unified theory of high availability.

Better late than never...