11 months, briefly.  

A lot happened this year.  I learned a lot of stuff, but not really that much if you consider how much there is to learn.  I want to write down a bit about everything I’ve been doing..

Maybe I’ll work backwards.  So most recently, I was finally allowed to make maps for Scourge of War.  So I dug out my dusty random map generator that I started back in August 2011, and worked on it basically nonstop for about two weeks.  Get up at 7, work on it til midnight (with breaks to go to class) and repeat next day.  The result?  We have a bunch of cool new maps to play Scourge of War on.  As I’ve said before, I’m a terrain junkie.  A lot of those hours I spent were wasted (?) flying around my maps just looking at them, the way I look at terrain while riding in a car or on a plane or in a train.  Terrain is cool.

It was nice to have classes during these two weeks, since class time — at least when it’s boring — is when I have a lot of my ideas.  Because I’m stuck in a chair with paper and pencil, and I think of algorithms for generating creeks and roads and such.  The map generator code is weird.  Most of it is C#, and then there are a lot of supporting python scripts and batch files to combine everything together.  The C# is broken into different modules, for instance there’s a heightmap generator that makes hills and creeks, and it has to return where the creeks are so the terrain and texture generators can put blue there — and the heightmap generator takes the locations of the roads so it can make them sunken.  So each module has an input data set and an output data set.  Much of the code is generalized stuff — functions that apply geometry to make stuff on what are basically bitmaps of varying sizes.

I didn’t do each piece with performance as a priority.  I have a log file that gives me detailed info on how long everything takes, so when I have time to work on performance, I pick the biggest thing in the logfile and address that.  The biggest one was generating a bunch of 4096×4096 (2d) perlin noise maps.  Those took a few seconds each if I remember right.  So I made a little cache wrapper thing that stores them in files, and just grabs as many as I need from a set of two or three times as many as I need, so they don’t have to be generated on the fly.  No one’s gonna know if the random greenish texture splattered on one 5×5 mile map is in the same layout as the brownish texture splattered on another map.  The important thing is that they look good, and on a given map they don’t repeat which really makes it look more natural.

The current performance issue is the generation of creeks, which is not a simple problem in general.  My current implementation was one of the first things I did for the generator, and the results are OK, but not ideal.  If I get time I want to redo the creeks to be more realistic, and make sure performance is good next time.  But tracing creek routes on a 7.5^2 mile heightmap just isn’t something that can be done instantly, I don’t think.   (I need a bigger heightmap than the area used so the creeks can flow in and out along the edges.)  It only takes a few seconds so it’s not that bad.

Then there’s roads.  My first road algorithm made unique-looking road networks but they were not very useful in game since there were a lot of isolated tree-like road areas.  And there was no easy way to align the fields with the roads.  So I redid that as a randomly noisy grid, that makes it easy to split up the inside areas into fields, and makes the roads useful in-game.  I’m not happy with it, but it looks good and plays well.

(one of the random maps)

 

Before the random map generator, I decided to do something in C++.  I rarely do anything in C++ because I have all this library code I’ve written in C# and so many projects already released that I can improve with minimal resistance.  But I wanted to practice C++.  I sort of know how memory works in C#, and C++ is relatively straightforward in comparison since you have to do everything yourself.  You can’t really understand your C++ code without knowing how you’re using memory.

So to start with I made something with minimal effort that was useful — a TCP client/server app where the server runs on my brothers’ minecraft server, and we can connect using the client and send it messages to, for instance, teleport our character around, or swap our inventory between different worlds.  I made it do just one thing, sort of hackily at first, but it worked.  And at that point, if I wanted to improve it, I had to write C++ since it would take more work to rewrite in C# than to continue writing C++.  So I implemented my GCSV parser and ini-file handler in C++.  (These two things are particularly important to me and are used in all of my C# projects, since I like to keep as much of my data in external files as possible, and I always like to change things from hardcoded to options when possible.)

The GCSV parser in particular forced me to decide how to handle memory.  I’ll start from the beginning.  A GCSV file looks like this:

~myfile
col1,col2
a,field_for_a
b,field_for_b

So when I read the file, it detects ‘~’ as the start of a new “GcsvTable” and uses col1 and col2 as the column names.  So I can say something like

var gcsv = Gcsv.ReadFromFile("myfile.csv");
foreach(var line in gcsv) {
    var field1 = line["col1"];
    var field2 = line["col2"];
    Assert.IsTrue(field2 == "field_for_"+field1);
}

So I can easily add columns and change the order of columns in my csv file without breaking my program.  My C# code for this is available in the GCSV library in my open sourced Utilities.dll code.  It’s awesome for me, I don’t expect anyone else to use it.  That code I’m still happy with even though I wrote it two years ago.  Oh, and since I githubbed this minecraft service I’m talking about as well, the C++ version (which is relatively barebones in comparison) is also available.

GCSV also supports a default or specified index for a Table, where I could say gcsv["a"] and get the first line in my example file.  This is lazy-instantiated since I usually don’t need it.  And there’s other cool stuff, like a GCSV file can have multiple Tables in it which is useful for instance in GCM where I have like fifty tables each representing an option when creating a battle.  It’s reasonably optimized in C# so the overhead of loading a typical Table is very little more than just the text itself.

Where was I.. oh yeah, the GCSV code in C++ forced me to decide how to handle memory.  I sort of knew how smart pointers worked, but hadn’t used them in anything important where I really wanted to know exactly how they worked.  With this, I did.  See, the GCSV file gets read into a Table object, which has a Header and a bunch of Lines.  Each Line has a reference to the Header — so you can pass a Line around and use it like a read-only associative array, even though all it is is a simple array of strings and a pointer to the header.  But the Line shouldn’t know about the Table, because in theory a Line could be passed off somewhere, and if it’s the only Line my code knows about anymore, the Table and the rest of the lines should be freed.  Likewise, the Header doesn’t know about the Table.  Anyway, shared_ptr is great and this all is pretty straightforward and it works and makes me happy.

With the GCSV code in C++ it was simple to set up the TCP server with a data file listing the minecraft worlds and the teleports for each world in gcsv files.  Anyway, I worked feverishly on this for a couple weeks, and we used it, it worked, great.  We keep a vanilla minecraft server since we don’t play enough to justify keeping a set of mods compatible with the game as it is updated.  Since my service just accesses the world files, and these almost never change, my stuff doesn’t really get out of date.  (The specific files I access haven’t fundamentally changed ever since we started our world in minecraft multiplayer beta.)

 

Before that, let’s see.. I spent the summer in New York, as an intern at Liquidnet.  It was a great internship.  Good people, minimal bureaucracy, a code base which — while big like they always are — was less of a nightmare than I would have expected.  (Granted, as an intern I wasn’t working with all or most of it, but I was exposed to a lot of it.)  Build/test infrastructure was pretty good I thought.  I was able to work on stuff for real people that I think might actually be used — my work was in UAT when I left, to ship when the in-progress system it was built for ships.  Informal work environment which I don’t imagine is typical of finance.  Lots of food.  Aside from students and professors it’s the first time I’ve worked directly with other programmers, and that was good.  I was happy.

I liked living in the city.  First of all, there are tons of people, so whatever you want to do you can find people doing it.  You don’t have to get in a car and drive several miles to find food or people.  I fully intend to live somewhere more densely populated when I graduate.

(the view from my window at Columbia)

During the summer I got some more exposure to F#, which I had dabbled in before, and finally actually used for “production” code.  I rewrote and expanded on a chunk of my GCM code base in F#.  A lot of that code really maps well to being purely functional, and the size of the code that I rewrote cut to less than one half what it was before.  F# is really good.  (It’s OCaml with .Net integration).

 

Before the summer I worked a lot on GCM.  Added a singleplayer campaign that by current count, 125 people have played, with 1000 battles recorded so far.  This was interesting because I had to take my GCM multiplayer campaign, and basically have a clone of it for each singleplayer campaign.  So everything I had written with “just one of these” in mind became “n of these”.  It wasn’t that difficult really, since I had the idea of “a campaign” right from the start, and just had to allow more of these.  So each campaign has its own index of regimental histories, which is one topic I want to talk about some time since it involves packing lots of bits of data (hundreds of thousands of “this regiment fought in this battle under this commander” entries) into a small amount of space with fast access for the one common access pattern — which is “show me all the battles this regiment fought in.”

 

I have a few other blog posts that are sitting as drafts, that I’ve been working on.  One of them is the history of my perspective of the Scourge of War / GCM scene.  Basically a narrative of the challenges the game faced in the multiplayer scene, and how they were addressed over the past two years.  But that story continues to get longer, perhaps faster than I write the history.  Either I’ll speed up, or just post a part 1, or it’ll never get posted.

Leave a Reply