Creating Planets

The next stage of Project Venice is to introduce the graphical element.  I'll be using Unity and their web client to do this.  I want players to be able to see the game world and Unity makes this easy.  It'll be an isometric view of the solar system.  Nothing too complex to start with.

I will be using Socket.io to populate the Unity web client with data.  This type of connection is open to everyone, so conceivably players could look at the game world without ever having to log in.  I'll have to look into that idea later on.

On the Unity side,  I am using UnitySocketIO, which provides a nice easy to use interface.  When the client starts up, it requests the data for the current star system.  The server grabs it from the database, as a JSON string, and sends it down to the client.  The client then renders the planets based on the data.

To make the planets I am using a Unity store asset called Perlin Planets.  It allows for the random generation of planets based on a seed value and some other options.  The long term goal of Project Venice is to have an explorable galaxy, so being able to generate planets in a procedural way is very important.  For the prototype, there will only be a few planets though.

One challenge I came across during implementation was that the UnitySocketIO library actually operates in a separate thread.  Unity is not multi-threaded, so I can not interact with it from an outside thread.  So when I receive data from the server, I can't just create the Unity objects.

To get around this, I put in a simple job system.  The SocketIO side will queue up a job to create a planet, then the main Unity thread will pull it off the queue and do it.  In my limited tests it seems to work fine.  We will see if it scales up once I start adding starships to the mix.

Read More

Dropping MongoDB for Redis

Up until now,I have been using the NoSQL database MongoDB for Project Venice.  It is very neat and easy to use.  It is almost perfect for my needs.  However, the more I thought about it, the more I was bothered by one feature MongoDB lacks:  transactions.  This is kind of a deal breaker for the game prototype I am developing.

Let's look at the scenario of a player buying an item off of a marketplace.  There are 2 main things that need to happen:

1)  Check and deduct item inventory

2)  Check and deduct player funds

This would involve two MongoDB documents.  While each of the two can be done atomically, they can't both be done together.  So stuff could happen in between.  This would be a bad thing and lead towards all sorts of problems.  MongoDB just does not support atomic operations across documents.

I started to research alternatives.  A traditional relational database ( mysql, postgresql ) would do the trick but I didn't really want to go that route.  This prototype is about learning new things, so I wanted to stick with NoSQL.  Redis and FoundationDB ended up being the two I seriously looked at.

FoundationDB has full ACID/transaction support but looked pretty daunting.  Redis, on the other hand, has transaction support ( but no rollback ) and looked very simple to use.  Keeping with the KISS philosophy, I went with Redis.  Implementing it was very simple and did not take long.  So now Project Venice is full on Redis-fied.  

Redis also has the advantage of being memory resident, so it is very quick.  The drawback is that it needs a lot of RAM.  I'm not anywhere close to needing to worry about that though.

Read More

Window to the World

My first milestone for Project Venice was to make a chat system.  With that complete, it is time to make my next goal.  Project Venice is heavily data driven and makes use of HTML to interact with that data.  An entirely form based game isn't going to cut it in 2014.    So, there will be a 'window to the world'. 

With this feature, you'll be able see the solar systems (only one in the prototype) and ships.   I am going to use Unity for this part of the project.  I was tempted to go HTML5, but Unity is better (quicker) to prototype with.  Basically, the Unity web app will use Socket.IO (web sockets) to get data from the server.  So all of the locations in the solar system will be displayed in the Unity window.  It'll be an isometric view, so think birds-eye but at an angle.  

Not only will locations be view-able, but you'll be able to see starship traffic between them.  This traffic will all be caused by player activities.  The main goal of this feature is to let players have a visible effect on the game world.  So if a new trade route develops, you'll be able to see traffic increase between the locations.   We need to crawl before we walk, so the goal here is to just have planets showing up and for the app to be integrated into the webpage.

Read More

Chat it up!

With Project Venice, I am setting concrete goals for myself in order to keep the project moving.  The eventual goal is a playable prototype.  My first milestone was to create a working chat system.  This involves people to be able to log in, and send messages to each other.  It sounds simple, but I had to create an account system, and get client to server communications working.  So a lot of base work for future systems had to be put in place.

To handle the chat system, I leveraged Pusher.  Pusher is a web sockets Publish/Subscribe system which makes it very easy to handle real-time communications.  Clients subscribe to the global chat channel, and on the server side I send messages to the channel and everyone gets them. Very easy.  I could have had clients send messages directly to each other, but that has its drawbacks.  I need to sanitize what is sent, so going through the server is a necessity.  Don't need folks sending XSS and other such nastiness. 

I also implemented a Title system.  Everyone starts out with the title 'Citizen.'  You can change your title, if you had more than one.  Right now the title is just used in the chat system.  That's another reason I need to go through the server, so I can add title information.  

You can check it out (and test it) here:  http://venice.duckdns.org/signin  There probably won't be anyone to chat back, but if you see your message in the window, it worked. 

 

Read More

Lots of Money

I was working building out some Project Venice data structures, specifically money, when it occurred to me that an Integer value would not be enough.  It would only allow players to have ~2 billion bucks (  2,147,483,647 exactly).  While that is more than enough for the prototype, I figured I better solve that issue now by moving to 64bit integers.  That will give us a max value of 9,223,372,036,854,775,808.  I think that should be enough :) 

Javascript does not really support 64bit numbers, which will make doing things like addition and subtracting difficult.   By default, mongoose ( the ORM adapter I am using for MongoDb ) also does not support a 64bit integer data type  Luckily, there is a workaround with mongoose-long.  So I went ahead and added that to my package.json file then did a npm install ( or you can do npm install mongoose-long).

Next, I adjusted my database schema and added:   money:mongoose.Schema.Types.Long

Now you can pass in really large numbers to the money field, hooray!  We're not quite done yet though.  The Long datatype is actually made up of a few fields and looks like this:  money: { _bsontype: 'Long', low_: 1942892530, high_: 2874 }.  That's not very useful, so we need to use toString() on it to get it to display as a readable number again.

If you are not using Mongoose, I'd suggest node-bigint.  It is pretty to use:  

var m = '1000000000000';

var lotsofmoney = bigint(m).add(10).toString();

With these options, Project Venice now supports a whole lot of money.  What will you do with it all?  Well, that will be the fun part :)

Read More

Concurrency

I've reached the point where I am ready to do some coding for Project Venice.  I'm starting with the basics: client to server communications.   Node.js is a bit different the way it approaches things.  Unlike other programming languages, it is not very sequential.  Instead,  it's all about doing work when a call has completed, rather than just being next in line.  

My client (the browser) will communicate with the server (Node.js) using AJAX.  So what happens if a user double clicks a button and one request arrives before the previous one is finished?  That is the problem I am most worried about.  In fact, this issue may also affect my Google App Engine projects, but to a lesser extent.  In any case, it is time I solve this problem for good.

Here is a scenario example from the same user who is attempting to purchase some in-game good twice:

Request A: Check Funds

Request B: Check Funds

Request A: Subtract Funds

Request B: Subtract Funds

When we get to the last step, there may not be enough funds left since it doesn't know a previous request subtracted funds.  That is a big problem.  So how do we go about ensuring requests don't interfere with each other?  MongoDB, which is what I am using for this project, doesn't have a concept of transactions. Nor does it really let you lock anything specific down.  But is does provide a method to avoid the above issue.

MongoDB provides a findAndModify() function which lets you query and update a document atomically.  That means it will happens at the same time and other db operations won't happen in between.  Mongoose, the Node.JS ORM, provides wrappers for these functions: findOneandUpdate, findOneandRemove, findByIdAndRemove, findByIdAndUpdate.   An example from https://groups.google.com/forum/#!topic/mongoose-orm/5b94sq-pjsg:

var query = model.findOneAndUpdate({

      _id: 123,

      balance: { $gt: 10 }

    },

    {

      $inc: { balance: -10 }

    });

Using these functions,  I can avoid some concurrency issues.  However, these are only good for a single MongoDB document.  What happens if I am accessing multiple documents?  That is a post for another time. 

Read More

Logging In

One aspect of my previous projects which I liked was that you could log in using existing accounts on Google, Facebook, and others.  It is convenient for the user and it saves me from managing credentials.  Sounds like a win-win to me.  The only downside is that people can get confused which account they logged in with before.  I think the upside outweighs the confusion issue though.  I want the barrier of entry to be as low as possible.

Sites with Benefits leveraged Janrain for authentication.  I'm going to go a different route with Project Venice, Passport.  This solution was built for Node.js and supports a large number of authentication providers including Twitch and Steam.  To start with I am just going to use Google.   Well, that was the plan at least.  Instead, I was greeted with:  OpenID auth request contains an unregistered domain.  It seems Google has depreciated the OpenID method of authenticating for new sites.

The 'proper' way to do this now is with Google+.  Luckily, there is a Passport Strategy for this as well, so it is easy to plugin.  On the Google side, this post explains how to get started.

Initially, I tried to create OAuth2 tokens using my hosts IP address.  Google did not like that for the redirect URI, as it wanted a hostname.  I don't want to buy one quite yet, so I went to http://www.duckdns.org and created one.  Google is now happy with that.  

With Google+ authentication working, I went ahead and hooked up MongoDB so I can actually create a user.  I don't really want to use any of the name information I get from Google, so players will have to create a username when they log in.  That means I will also have to develop a filter to prevent obscene names. 

You can log in an try it out :)  Let me know if it works or not.  You won't receive any spam as the database won't last long. 

http://venice.duckdns.org/signin

Read More

Project Venice: Milestone One

In order to keep on track with my new game initiative, Project Venice, I am going to break it down into smaller chunks.  The feeling of progress is very important to keeping a project going.  The end goal is just to make a prototype, which is still a lot of work.   So I've defined what my first milestone for the project will be.

Chat System.  What is a multiplayer game without a chat system?  I think this is a useful and reachable goal.  It requires a working login system, account system, and client - server communications.  As part of the chat system I really want to include configurable titles.  So what better time than now to add support for them?

E-Mail Verification.  As a part of the account creation system, I want to allow people to change their email address.  To do this, I need to verify the address exists which means I need a mail system. It's not the most exciting of projects, but one that needs to be done.  I do this for Sites with Benefits already, so I just need to port it over to Node.js.  

I've already made good progress towards my goal of a chat system.  I hope to have a demo of it all up and going very soon.  

Read More

Project Venice: A Home

Since I am taking a whole new approach with this game, it requires a new home.  I want to avoid getting limited by quotas this time around too.  There are a lot of solutions, like Amazon's EC2 and Heroku plus a myriad of other cloud products.  I've decided that this project will have a budget, so paying for hosting is doable.  This post will be down in the weeds :)

After looking around, I have settled on Digital Ocean.  I did this for a few reasons.  

$5 a month.  Their most basic plan is pretty affordable.  It should be more than enough for a prototype.  Plus, if you google it, you can find promo codes for $10 in credit.  So I'll actually have 2 free months to play around with it.

SSD Drives.  I probably don't need SSD storage,  but who can say no to that?

MEAN Stack.  They offer 'droplets' with the MEAN Stack built in.  That means less setup for me.

Signing up was very simple.  Although, I still needed to enter payment information and pay $5 despite my $10 credit.  That gives me 3 months worth for $5, which is still pretty good.  

I chose the smallest instance, which gives 512MB/1 CPU/20GB SSD/1TB Transfer.  After choosing an image with MEAN on it, I was all setup.  Using SSH I logged into the server and proceeded to update the server using 'apt-get update && apt-get upgrade && apt-get dist-upgrade.'  I really hope I need to upgrade to more a expensive plan someday, that would mean it is being played. 

The image comes with a MEAN website all setup, so I rm -rf'd it and decided to start from scratch.  I used the following command to set it up again.

mean init venice

cd venice && npm install

grunt

This started up the webserver on port 3000.  However, it threw an error when I connected to it:  Error: ENOENT, open '/opt/mean/venice/public/system/lib/bootstrap/dist/css/bootstrap.css'

To solve this, we need to install bootstrap-css:  bower install bootstrap-css

This command also throws an error:  "bower ESUDO         Cannot be run with sudo".  Since we are running as root in our image, we need to do:  bower --allow-root install bootstrap-css

Now we run grunt again and connect to the site, success!  

However, this still leaves us with quite a robust install.  I really want to start with nothing and build it up so I wiped it again.  Then I found this resource:  http://scotch.io/bar-talk/setting-up-a-mean-stack-single-page-application  After following those instructions I was left with a very simple install which is much more understandable.

Finally, I setup the firewall for my new system.  I found a great guide about how to do it.

You can check out the new site at:  http://107.170.182.216/

Next up, getting a login system working.

Read More

Comfort Zone

I am very attached to my routines and comfort zones.  It's probably one of my biggest flaws.  It's something I have been actively working at in my offline life, so I figure it is something I should also work at online.  Project Venice is a part of this effort.  It uses a number of technologies which I have no experience in.  As I researched them, I definitely felt some creeping anxiety.

MongoDB.  No SQL here, or Google's datastore which I have grown used to.  This is pretty simple though, so I'm not really worried.

AngularJS.  I'm so used to using JQuery, well no more of that.  This is a new framework which I will have to learn.

ExpressJS.  This is a web application framework, of which I have no experience.  The documentation scares me a little.

NodeJS.  I've gotten very used to using Python and App Engine's way of doing things.  The asynchronous nature of Node has me very nervous.  I'll need to change my way of thinking.  

Jade.  I usually write my HTML by hand.  Jade is a templating language which is supposed to make things easier.  It means I'll have to learn a new syntax.

All of these things are new to me.  I find myself thinking, "maybe I should just do this the old way."  No!  I must keep on the path, no matter how uncomfortable I may be at the moment. 

Read More

Project Venice

If you have been a reader of this blog for a while you probably know that I like to work on programming projects.  Sites with Benefits ( formerly Project Murdock ) was my last project and it is in a pretty good spot.  So I've been looking for something new to work on.  Inspired by my playing of Ascent, I want to revisit some game ideas I have had over the years and try to put together a prototype.

As it is not worthy of a real name yet, it will be called Project Venice.  The game itself would be a different take on the space trading genre.  Instead of taking the role of the pilot hauling cargo, you'll be in management.  Let's face it, hauling cargo is tedious.  The real fun comes in the decision making and trade route discovery.  Those are the aspects I want to concentrate on.

Another purpose of this project is for me to learn something new.  Instead of using Google's App Engine (like I do for everything)  I am going to switch things up.  Right now, the plan is to use a MEAN Stack and a hosted service provider.  It's not free, but cheap enough to make it worthwhile.  MEAN stands for MongoDB, Express.js, Angular.js, Node.js.  I have not used any of them before, other than a little project with MongoDB.

I plan on this being a series of blog posts, many technical, about the development of the game.  I don't expect to get to the fun stuff for a while though as there is much setup to be done first.  Determining a hosting provider is my next task.  

Read More