Dev Diary 026: Learning SmartFoxServer

warning: Creating default object from empty value in /home/buckman/public_html/neo/modules/advanced_forum/advanced_forum.module on line 492.
Matt Barton's picture

There's two problems with the current Outlaws game. One is that it requires joystick support, which while great if this were an Xbox 360 indie title--and, who knows, maybe one day it will be!--most PC gamers don't have a gamepad connected to their machine like I do. Secondly, even if they did, they don't have anyone locally around willing to play the game with them. Adding keyboard support as an option is relatively trivial from a coding perspective, so I decided to tackle the far greater problem first--online multiplayer.

My first approach was to purchase an asset from the Unity assets store that'd do all the heavy lifting for me. That product, called Fieldkit Readymatch, looked very promising. Unfortunately, I couldn't ever get it to work, and since its functionality depended entirely on their server, I also didn't have a way to know if the problem was on my end or theirs. Eventually, a guy from the Unity assets store contacted me and said they were refunding the purchase price of $100 and removing that asset from the store! So I guess it wasn't just me after all.

So then I turned to a product called Smartfox Server. It came recommended by a Matt Chatter who's a pro, so I figured what the heck. It's free and has a C# API, and has some tutorials that pretty much do everything I needed. However, wrapping my head around this stuff has pushed to my absolute limits. It's very difficult for me on a conceptual level, trying to keep in mind that there will be multiple instances of the game running on different clients...I know this all comes naturally to a lot of you guys, but for me it's maddening. I mean, I had a hard enough time mapping out two different control schemes for two players on the same screen!

I'm kinda pumped right now because I think I have all the functionality figured out to do what I need to do. Rather than go straight to hacking Outlaws, I did a preliminary scene. I now know how to set up zones, rooms, have players connect to those, and so on. My sample scene just has two blocks moving around, but the exciting thing is that players control their own block--and can see the other block when it moves. I still need to work out the intricacies of "interpolation" to deal with lag, but it's still a huge step forward.

My next headache was figuring out how to setup my solar system so that both players will see the same thing. I thought about it for awhile, and reckoned that really all I need to make this happen is to generate all the variables first, then just send them in a package to all the players. I got really confused, though, since there seems to be several fundamentally different ways to go about it.

One commenter suggested doing this stuff server side, which sounded like a plan to me. However, as far as I can tell, the only way to do that is with a server side extension. The example videos show this being done in JAVA. Oh, boy. To make matters worse, my internet connection has been absolute shit lately, and I wasn't able to access the Netbeans site or the Java site...Or the Smartfox Server docs for most of the day.

I decided that's a lot of trouble to go through, so then I started wondering if I could just have the first player's client do it. The game rooms assign each player a playerID, which always starts at 1, so it seems straightforward enough. I can check to see if it's player 1, calculate and put all the numbers into room variables, which I can then send out to the other players. I was able to do this successfully with a random number test, so in theory...If it works for one random number, it should for an array of them.

I'm not quite sure what's going to happen if Player 1 drops...Will the room variables stay the same or change? I guess for now I'll just automatically end the match and end players back to the lobby if that happens. I really want to have four players on screen eventually, but might have to wait on that until I get the system figured out. Even though this approach seems to work, I don't like it much, so I'm going to try again to see if I can make some headway into doing this stuff server side via a room extension.

I'm really hoping I can get this server stuff figured out, because I can imagine tons of fun possibilities. I did find six basic tutorials on Unity and Smartfox, but, boy, could I use some more. It seems odd that there isn't more stuff out there on this. Surely it's a common enough thing for indie developers.

Comments

Matt Barton
Matt Barton's picture
Offline
Joined: 01/16/2006
Yay

Looks like I'm in business with the server side extensions. It took a long time for me to try to get it to work, but I'm now able to set up some stuff on the server and access it from the clients. I don't even understand 10% of what I'm doing, but if it works, it works.

Now I just need to set up the server stuff to do all my solar system work and I should be ready to roll.

n/a
Shawn Delahunty
Shawn Delahunty's picture
Offline
Joined: 08/01/2011
Luck I am wishing you, for grave and difficult, your task is

This will be cool Matt, so stick with it. However, as you've certainly figured out by now--you are "...jumping in at the deep end of the pool." More like the North Atlantic really, with icebergs, tidal waves, and sharks with frickin' LASER BEAMS attached!

You are going to hit a lot of "concurrency issues", particularly when dealing with varying lag amounts between players. (Did player #3 hit 'shield' before or AFTER that lagged shot from player #1 would have hit? Did player #2 thrust out of the way before or AFTER the bullet would have "normally" impacted?) So my wholly unwarranted advice is: take your time, and pace yourself with the development--you might burn out in frustration otherwise.

Also, with regards to deciding a "hit" or "near hit" (the phrase "near miss" doesn't make sense--wouldn't that be a glancing hit?), you'll eventually get down to the "angels on a head of a pin" level of niggling concerning timing/lag adjustment. (Been there, done that, have the grey hairs to prove it.) Step back, make a blanket "Policy Design" decision, and then let that guide your "Technical Design" decisions for implementing the "wiggle thresholds".

(Not that you care, but the game does launch and run just fine inside of WINE on Linux--but it isn't detecting the joypad. I haven't had time to figure out if this is a WINE/USB configuration issue, or the fact that I'm using a Logitech Dual-Shock gamepad. I'll play with that a bit more.)

-S

n/a
Chris Rasmus (not verified)
Great that you're working on

Great that you're working on multiplayer! And glad to hear you're going to put keyboard support [back] in.

I don't understand a lot of the terminology you're using to describe your problems with getting multiplayer running and I don't expect you to explain them to me. Server side extensions, room variables? But I do I do feel your pain. For my attempt at a multiplayer roguelike (written entirely in c/c++ using OpenGL) my brother suggested I tackle multiplayer first before adding any features. It took me a long time googling winsock tutorials, making 2-3 attempts at basic server-client programs, etc., before I was finally able to incorporate the code and get a server and client working. Even now I have to go back and rewrite my code because it's just a buggy hack in its present state. I start my server, players connect and can see each other on their screens and as they move around but there are some weird things going on and a lot of times players just lock up, unable to move. It's been fun though.

Speaking of server-clients, and to elaborate...I would have the server program create the Star System. How many planets, moons, distance from sun, etc. I imagine it takes up very little memory. Then when each player connects you send that info to the player and the client creates the Star System from it on their end. When each player connects you also need to send that player a list of all players currently connected, and you need to send every player already connected a message indicated there's a new player. Likewise when one player leaves/disconnects all remaining players need to be notified. I imagine you've already thought of these things but I just wanted to throw it out there just in case.

Good luck and I'm really looking forward to your next playable version!

Take care.

Matt Barton
Matt Barton's picture
Offline
Joined: 01/16/2006
HI, Chris, and Shawn, thanks

HI, Chris, and Shawn, thanks for the comments.

There is a few lines of script that ship with Smartfox that deal with interpolation. I'm hoping (probably wrongly) that it won't be a big deal for my game, since there is not much data being passed back and forth, but we all know what a headache lag can be. There are built in functions for interpolation and extrapolation, but it's difficult to experiment with this stuff before I've seen it in action. From what I can tell, lag will be a real issue for players in other countries, which is actually a pretty good possibility here. I guess we'll just have to cross that bridge if and when we get to it.

I'm also wondering about the best setup for the actual game. I'm running the server myself, so obviously there will be severe limits on the number of rooms (concurrent games) and players. It has to be light enough on my end that it's not going to kill my PC or network performance. If it does get really popular, though, I'll start looking into other solutions. I'm sure there must be outfits out there who host this stuff for a fee.

For the first iteration, I'm going for "dead simple" multiplayer. I have one zone and one room, basically. You connect to the server, and then it tries to put you into the room, which I'm limiting to 2 people for the time being. The game only starts once two people are in there. After the game, I'll boot everyone out and start it over.

At this point, a number of design decisions pop up...Just a matter of figuring out what is the most fun and realistic.

How many players do you want in a room? It might be more fun if there are four simultaneous players rather than just two. I could, perhaps, have a minimum number of players (2) and once they're available, start a little timer to see if anyone else jumps in. Or I could give people the option to join games in progress...Or just add a few more rooms so you'd have more matches to choose from. I could also allow players to "host" matches, setting the min. and max number of players and so on. Lots of possibilities. Should the match be over when you die, or should you be able to respawn after a given period? Obviously there's no right and wrong answers here; just whatever makes for the most fun.

I also notice some documentation for interfacing with databases...Running a database is probably too big of a pain just for this, but I might look into it anyway if the game takes off. That'd give me a way to record and display data on people's careers or achievements or whatever. I'd even toyed with the idea of using AA's database for logging in to the game. The only real downside to that is of course not everyone is registered here. I'm not sure it's right for this game.

In any case, I think it's probably best to KISS at this point. Focus on getting a multiplayer prototype built that works, and expand from there.

n/a
Matt Barton
Matt Barton's picture
Offline
Joined: 01/16/2006
Updates

Just to give ya'll and update on my progress with this...I've been working on it obsessively pretty much non-stop. Tackling the network stuff has been my greatest challenge to date, no question about it. The documentation is sparse and not very helpful to non-experts like me, so it's been agonizingly slow, trial-and-error type stuff.

But here's where I am at the moment...

The players log in (right now that amounts to just inputting a name), select their ship, and enter the solar system. You can see the other guy's ship moving around, rotating, etc. All the planets and movements are synced. Actually, there's a noticeable delay if you put them side by side, but it makes no difference when you're playing the game. If one of the games freezes (if you pause it by dragging the window, for instance), when it returns the ships/planets will zip back into step. I'm not quite sure what I'll do about that yet, since it is obviously way to cheat. I've fished around for some kind of answer...This seems like a problem that would crop up for anyone, so hopefully there is some decent docs or solutions out there already.

Unfortunately, there's still a LOT of work left to do before I'll have a prototype ready...I need to implement all the projectiles, the various ship models, and of course work out how many people I can let play this simultaneously. There's still some other kinks to work out, too. But I feel like the heavy lifting is done; unless more unexpected tech problems rear their ugly heads (of course they will), I think I have everything figured out technically and just need to implement it at this point.

One big hurdle was updating the server...I originally had this happening in fixedupdates, which happen many many times per second. That was obviously draining the system terribly, so I opted to just update it every tenth of a second. That seems the magic number for something that looks smooth (to me, at least).
One big issue with this is that there is constant movement on everything. This would be a snap if the planets were in a fixed position!

If I survive this process, I'm kinda toying with the idea of doing a Smithereens! type game just for kicks. It'd be a hoot to have the original voices. I definitely went wayyyyy overboard with this game! I should have never taken on something this brain-bustingly complex with so little experience with unity.

I tabled the idea of doing stuff server side, mostly because that requires JAVA. That's a whole different language, and I kept running into issues with features in Unity C# that I couldn't find an equivalent for. I'm sure it's all there if I could figure it out, but I'm not convinced it's worth the effort at this point. In retrospect, I really should have found a server that had extensions you could program in c#. I bet there is probably something a bit better integrated.

Hopefully I'll have something to show you guys in a week or so.

n/a
Rowdy Rob
Rowdy Rob's picture
Offline
Joined: 09/04/2006
How does the Server work?
Matt Barton wrote:

Hopefully I'll have something to show you guys in a week or so.

I've been following your Dev Diary with a large sense of amazement. Server-based two (or more) player gaming over the Internet? That sounds extremely complicated! How's Matt going to pull that off?!?!?! But, you keep topping yourself somehow.

How is this server stuff going to be handled? Are you going to set up a dedicated server for us to play on, or are we, the downloaders, going to need to set up servers? Is there a separate program for client and server, or is it all handled in one program?

Just curious. Looking forward to the next update.

Matt Barton
Matt Barton's picture
Offline
Joined: 01/16/2006
Hi, Rob. The server stuff is

Hi, Rob. The server stuff is kicking my butt! Every time I think I have it figured out, I run into problems and have to rethink my approach to it...

I'm stuck now trying to figure out how to handle bullets. I have the ships and planets all synced nicely, but projectiles are a bear. There are, of course, lots of them to deal with, and that's a lot of info going back and forth. My initial approach was to use arrays and a foreach loop to do it, but ran into issues with that...If you fire three projectiles, let's say, and #2 explodes, and then you fire another one, my engine got confused and would do crazy stuff. I did a lot of research on it, and came up with lists and dictionaries. My plan (yet to be implemented) is to assign each new projectile a key in a list and hopefully use that to keep the coordinates straight for each one, maybe removing the dead ones from the list. Such a pain. I keep getting the feeling that there's a much simpler and more direct way to do this, and I'm just needlessly making it more complicated than it needs to be.

My plan for the moment is just to do whatever I can, no matter how clunky, just to get it to WORK. If I can just get the game running at a playable framerate, then I'll go back in and try to smooth things out and work out more efficient ways to do things. But just the conceptual framework is driving me insane. I tell you one thing, I have an all new respect for people who do this network stuff. I never dreamed how complicated it would be, not just technically, but conceptually..."Spawn" a remote player...make sure your local player is transmitting the right info...but don't overwhelm the server...

On a positive note, I'm definitely doing the HARD stuff by having a real-time game. After this, a turn-based thing would be a snap.

n/a
Rowdy Rob
Rowdy Rob's picture
Offline
Joined: 09/04/2006
Matt Barton and the Temple of Doom
Matt Barton wrote:

On a positive note, I'm definitely doing the HARD stuff by having a real-time game. After this, a turn-based thing would be a snap.

When haven't you taken on an over-the-top hard challenge?!?! It seems to be your M.O. :-)

The amazing thing is seeing you escape the jaws of doom every time.

Matt Barton
Matt Barton's picture
Offline
Joined: 01/16/2006
Well, after four agonizing

Well, after four agonizing days of morning-to-night trial and error--I do mean AGONY--I have at last discovered why I'm having so much trouble. I kept thinking it was some issue with threading, arrays, indexes, etc...All along, though, I was just hitting the maximum and the clients/servers weren't receiving and transmitting all the info, but leaving a chunk out every time.

Apparently, I'm sending too much data to the server. I can have moving planets or projectiles, but not both, apparently. At least unless I can figure out "string protocol," which is offered as a remedy to this problem on the Smartfox site. I plan to do some research on that, next, to see if it's anything I can realistically hope to implement.

Unfortunately, I hacked my code to spaghetti trying to figure out how to make it work, so there will be a lot of clean-up necessary now, too. Burnout is looming.

n/a
Matt Barton
Matt Barton's picture
Offline
Joined: 01/16/2006
Okay...I almost hate to say

Okay...I almost hate to say this for fear of jinxing myself, but as of this moment everything is working. We got moving planets, projectiles, ships, the whole shebang.

I was able to workaround the problem by sending the planet information as a room variable rather than a user variable. Apparently, you can only cram so much into a user variable.

I did some research into alternatives to sending SFSObjects and SFSArrays, such as JSONs and "raw strings." Unfortunately, what I found was sparse and not tailored to a C#/Unity guy. It appears that most people using Smartfox are doing so with Java or ActionScript; Unity is a recent addition and not nearly as well supported.

From what I can see, the main idea behind SmartFox is to program the bulk of your game's logic using JAVA, and then attach that code to your game in the form of room "extensions." That'd be great if I knew JAVA, but I don't. While the syntax is similar to C#, it's different enough to confuse me. A bigger issue, though, is the totally different libraries and "name spaces." To really be effective, I'd need to know JAVA well enough to do all the "heavy lifting" server side, and just send out coordinates and variables to the clients. I'm not entirely convinced the savings would be worth it for a simple two-player game, but it'd be an obvious necessity if you were trying to do anything really ambitious with lots of synchronous users.

At any rate, hopefully I'll be able to get something uploaded today. I'm stripping out all the extra features (like extra ships and weapons) for now for the sake of proving the concept.

n/a

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.