Unicode support in MySQL is ... 👎

Posted by

For the last few days, I've been getting some strange error reports from the War Worlds server. Messages like this:

java.sql.SQLException: Incorrect string value: '\xF0\x9F\x98\xB8. ...' for column 'message' at row 1
   at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1078)
   at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4120)
   at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4052)
   at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2503)
   at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2664)
   at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2815)
   at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2155)
   at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2458)
   at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2375)
   at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2359)
   at com.jolbox.bonecp.PreparedStatementHandle.executeUpdate(PreparedStatementHandle.java:203)
   at au.com.codeka.warworlds.server.data.SqlStmt.update(SqlStmt.java:117)
   at au.com.codeka.warworlds.server.ctrl.ChatController.postMessage(ChatController.java:120)
   . . .

Now, that string which MySQL complains is "incorrect" is actually the Unicode codepoint U+1F638 GRINNING CAT FACE WITH SMILING EYES, aka 😸 -- a perfectly valid Emoji character. Why was MySQL rejecting it? All my columns are defined to accept UTF-8, so there should not be a problem, right?

When is UTF-8 not UTF-8?

When it's used in MySQL, apparently.

For reasons that completely escape me, MySQL 5.x limits UTF-8 strings to U+FFFF and smaller. That is, the "BMP". Why they call this encoding "UTF-8" is beyond me, it most definitely is not UTF-8.

The trick, apparently, is to use a slightly different encoding which MySQL calls "utf8mb4" which supports up to 4-byte UTF-8 characters.

So the "fix" was simple, just run:

ALTER TABLE chat_messages
   MODIFY message TEXT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci NOT NULL,

And so on, on basically every column in the database which could possibly include characters outside the BMP. But that's not enough! You also need to tell the server to use "utf8mb4" internally as well, by including the following line in your my.cnf:

​character-set-server            = utf8mb4

Now presumably there is some drawback from doing this, otherwise "utf8mb4" would be the default (right?) but I'll be damned if I can figure out what the drawback is. I guess will just moniter things and see where it takes us. But as of now, War Worlds support Emoji emoticons in chat messages, yay!


Abandoned stars and new players

Posted by

It seems to be very common that people will install a game, play it for 5 minutes, decide it's not for them and then uninstall it (or just never play it again anyway).

For many games, this isn't a problem. But not so in War Worlds: if a player creates an empire, they'll be assigned a star in the universe, and then that star will be occupied forever - or at least until someone else comes along and wipes it out. The other problem it causes is that people will tend to start very far away from active players, which makes for a rather boring game.

What is an abandoned star?

Before we can do anything about them, it's a good idea to define exactly what an abandoned star is.

Basically, I have defined an "abandoned star" as any star where the empire has not logged in for 2 or more weeks, and this is the only star they have colonized.

Looking through the database right now, there are 5,887 stars which match the above criteria, which is quite a lot!

What to do about abandoned stars?

The next question is, what do we want to do about them? Before today, we did nothing: they just sat there waiting to be attacked and colonized by someone else.

But starting from today, abandoned stars will be elligible for new players to start on.

There's a few extra criteria before we'll start a new player on an abandoned star, however. The main one is that an abandoned star must be > 200 parsecs from any non-abandoned stars. Many of the abandoned stars in the game are surrounded by established players, and it wouldn't exactly be fair to star a new player off in the deep end like that.

So now, when a new player starts the game, we first look for an abandoned star more than 200 parsecs from an existing player, and only if we don't find one will we star the new player on a brand new star.

My hope is that, over time, the game world will fill out a bit more closely and we won't have vast spances of space where nobody is playing any more.

Advertising an Android game: Facebook vs. AdMob

Posted by

First, a disclaimer: I'm not - by any stretch of the imagination - an expert in these matters. It's quite possible that I could've got better results with more time invested. But I still think this is a useful comparison because, let's face it, not many people actually do have the time needed to invest in a proper online advertising strategy. Especially if you're just a one-man-team like me!

So with that out of the way, I want to describe my experience using both Google's AdMob advertising platform and Facebook's platform for advertising War Worlds. My goal with the campaign was basically to get more signups in the game, a metric which I can measure directly by looking at the number of new signups in the backend (it's not so easy to track where the signups come from [not impossible, just not easy], so I decided ran both campaigns separately).


As I said, very unscientific, but the ad I ran on Facebook was the one on the left. The text for the ad was "A new 4X MMO for Android. Explore, Exploit, Expand, Exterminate!".

I decided to target it at people in the U.S. and Australia who "like Mobile phones, strategy games, Massively multiplayer online games, Strategy games or mobile phones."

I gave it a budget of $20 over three weeks (i.e. just over $1 per day).


AdMob is Google's mobile ad network. You create an ad and it displays in any app which use AdMob's SDK. I used the ad on the left, and while the Facebook ad landed users on the War Worlds home page, this one landed users on the Play Store (this is a requirement of AdMob ads: I would have preferred clicks to go to the website here as well).

For this ad, I targetted only Android devices which support War Worlds (i.e. Android 2.3+). There's no fine-grained filters like Facebook ads (so I couldn't, for example, show it only to people interetested in strategy games). I set the budget to $1 per day.


To my eyes, the results are conclusive: Facebook was a complete waste of money, while AdMod is making a noticeable difference in active installs.

The Facebook ad, over the course of about 3 weeks and $20, garnered a total of 61 clicks. According to the "full" report that Facebook provide, I got 94,173 impressions. That's a click-through-rate of 0.065% and cost per click around $0.24.

Looking in the game's backend, there was no bump in new signups to the game at all (really, even if every one who visited the site signed up, that's only 61 potential signups, which -- over three weeks -- is not even a blip). Even though the Facebook ad was seemingly targetted very specifically to people who were actually interested in my genre of game, the actual number of clicks and actual signups wasn't even noticeable. If I maybe had money to burn, I could double or triple the daily spend and see some results, but for a game like mine, where the budget is already pretty razor-thin, it just doesn't seem worth it.

The AdMob ad, on the other hand, for a similar per-day budget, had quite a noticeable impact. Below you can see the impressions I got over the last few days:

The impressions seem to peak while Google figures out the best number of impression to get the number of clicks I want. So the first two days were slightly above $1, but then it plateau'd and the following days all cost pretty much exactly my $1 budget. But you can see, 236 clicks over only 9 days. Click through rate is an order of magnitude higher at 0.32% and cost per click is only $0.04.

And in terms of actual results -- i.e. signups within the game -- well, I think the results speak for themselves:

The yellow line there is the number of new empires created on that day and you can see the big jump after the ad started running. The red line is "7-day active" empires (that is, users who have logged in to the game in the last 7 days) and the blue line is 1-day actives (i.e. the number of users who have logged in in the last 24 hours). You can see the obvious jump on the 11th, when the ad started running. It's only been running for 9 days now (compared to the Facebook campaigns 3 weeks), but if this trend continues, I seen no reason to stop this campaign after the three weeks.


The conclusion seems pretty obvious, at least for me, in my particular situation. Facebook ads do not work, and AdMod ads do work. Maybe I could have spent more time tweaking the Facebook campaign to get better results, but when "time is money", why would I spend time on that when I can spend the same time and get orders of magnitude better results with AdMob?


So I've been informed that Facebook actually does support mobile-only ads. Putting in the URL to the app on the Play Store automatically triggers an "app install" mode which displays ads on mobile Facebook. I suspect this will be much more lucrative, so I'm going to give it the same $1 per day budget and give it a go for a few days. Expect a new blog post in a week or so with the results!


Posted by

In the next update of War Worlds, I'm planning to add a "wormhole" feature. Here's a brief overview of how I see wormholes working in the game.


You'll build a ship, called a wormhole generator, which will be a single-use ship (and quite expensive). Then, to actually create the wormhole, you'll select a location on the starfield for the wormhole generator to travel to and deploy the wormhole.

The ship will be quite expensive and time consuming to build, and there will also be restrictions on how close you can deploy the wormhole to other wormholes and also to stars. The way I see it, it should not be possible to build more than one wormhole between any single group of stars (see the mockup below for what I mean).

The wormhole will then appear on the starfield for anybody to see, and you can send ships to it like you send them to stars (i.e. via the move command).

Using the Wormhole

Below are some mockups for how the wormhole will look in the game, once constructed and deployed.


On the left, we see the starfield screen, with a wormhole selected. I'd love to have the wormhole itself animated and rotating, I guess we'll see how my art skills are :)

On the right, you see the "detail" screen for the wormhole. In the top-left, it shows you the destination of the wormhole, how far it is, how much it costs to send a ship and so on. In the top-right, you have various actions for the wormhole itself. You can change it's name (this is used to identify destination wormholes) and you can change the wormhole's destination (more on this in a minute).

On the bottom left, you have the list of ships that are currently at this wormhole. When you select a ship, on the right you'll have the usual options, like merge, split, move etc. You'll also have a special "Enter Wormhole" button that will cause the selected ship to enter the wormhole.

Wormhole Destination

The "destination" of a wormhole is another wormhole. When you first deploy a wormhole, the wormhole will have no destination and you won't be able to send any ships through it. When you click on the "Set Destination" button, a list of all the wormholes own by you and your alliance is brought up, and you can choose which wormhole will be the destination from that list.

Wormholes are one-way, so the destination wormhole may have an entirely different destination. In this way, your alliance could build a "highway" of sorts, with wormholes in a giant loop through all your empires.

Changing the destination of a wormhole takes time, and costs credits. When the wormhole is first deployed, setting the destination is instant, but after that, each new destination takes longer to "tune" (e.g. the first change takes 2 hours to tune, the next change takes 4 hours, 8 hours and so on). If you leave a wormhole tuned to a single destination for 7 days without changing, then the tune time resets to 2 hours. This is to encourage you to set up a coherent network of wormholes and plan ahead, rather than just setting them up willy nilly and changing them as you please.

Sending a ship through the wormhole will cost credits. It will be a fixed cost per ship based on the distance to the destination (longer jumps will cost more). It'll still be cheaper to use the wormhole (and obviously faster, too) than to move directly.

Wrap Up

Now, this sort of update takes time. Don't expect this in the next week or two :) In the meantime, I have a few bugfixes that I want to push out.

One important one is a change to the way ships from empires in the same alliance work. Currently, if you belong to the same alliance, and you send ships to each other's stars, they'll attach each other. In order for wormholes to be usable by the whole alliance, however, that's going to have to change. So expect an update shortly which will stop ships from the same alliance trying to attach each other.

Ship Upgrades!

Posted by

Today's update is a fairly big one in terms of adding to the game. I've added the ability to upgrade your fleets. Currently, there are three upgrades available, for three different ships. To access the upgrades, you visit the "Ships" tab of the build menu, where you will now see "existing" ships as well as "new" ships:

Tapping on one of your existing ships, you'll get a menu of the upgrades you can choose. Currently, each kind of ship only has one upgrade available, but in the future we may add some more. Choose the upgrade you want and click "Upgrade".

The build will proceed like an upgrade of a building. The main difference is that ships are available for upgrade from any colony (as long as it has a shipyard), so you can build a fleet of ships on one colony and upgrade them on another.

So what upgrades can you build?


As you can see above, the scout has an upgrade called "Cloak". Scouts with this upgrade are invisible to enemy radar, but more importantly, they can sit on an enemy star undetected. If you set the stance of the scout to "netural" or "passive" (to stop it from trying to attack), then enemy ships will not be able to see it and you can send it to spy on your neighbours.

This is great for seeing what your enemy is up to, even on stars that are outside the range of your radar.


The boost upgrade is available for fighters and, when actived, halves the ETA of the fighter. So if you send a fighter to another star, and it initially says it will take 2 hours, use the "Boost" button to halve that to 1 hour.

Boost is a single-use function. Once the fighters arrive at the destination, the boost is lost and you'll have to upgrade the fleet again to get another boost.


The final upgrade in today's update is called "Cyrogenics" and it applies to the colonyship. If you upgrade a colonyship with this upgrade, then instead of depositing 100 coloniests on a planet, it will deposit 400 colonists (or the planet's population congeniality, whichever is less). This'll give your new colony a bit of a boost when starting out.