Game Design, Programming and running a one-man games business…

Writing your own engine for 2D games in C++: basic considerations

So with all the current stuff in the news about Unity/Improbable and the counter offers from unreal, and the big epic vs steam thing going on… I think there are probably a few developers out there who think to themselves… ‘I wish I *did* have my own engine, then I wouldn’t have to worry about ANYBODY else’s code. The problem is… if you are used to unity or similar systems, you might have no idea where to start right?

The windows basics:

To have an app that runs under windows you basically need just two functions, WinMain and a Windows ‘procedure’ function that handles windows message. A stripped down winmain looks like this:

int APIENTRY WinMain(
                     HINSTANCE hInstance,
                     HINSTANCE hPrevInstance,
                     LPSTR pszCmdLine,
                     int nCmdShow)
MSG msg;

gInstance = hInstance;
GetGame()->InitApp();

//main program message pump loop//////////////////////////////////
while(1)
{
if( PeekMessage( &msg, NULL, 0, 0, PM_NOREMOVE ) )//any for us?
    {
    if( !GetMessage( &msg, NULL, 0, 0 ) )//if so get em
        {
          return msg.wParam;
        }
    else
        {
            TranslateMessage(&msg);
            DispatchMessage(&msg);
        }
    }
else
    {
        if(BActive)
        {
            GetGame()->GameProc();
        }
        else
        {
            Sleep(1);
        }
    }
}
return msg.wParam;
}

The only two exiting parts are InitApp() where I create all the Directx3D stuff, and load in any data I need, and GameProc() which is basically the games main loop. The window proc looks like this:

LRESULT CALLBACK WindowProc(HWND hWnd,
                                 UINT uMsgId,
                                 WPARAM wParam,
                                 LPARAM lParam
                                 )
 {
 switch (uMsgId)
     {
     //main switch statement for handling messages
     case WM_DESTROY:    //end the application
         PostQuitMessage(0);
         GetGame()->ReleaseResources();
         return 0;
         break;
default://default behaviour
        return DefWindowProc(hWnd,uMsgId,wParam,lParam);    
      }
}

Again the only exciting thing there is ReleaseResources() which basically closes down all that directx stuff and releases all the textures and any memory I allocated.

meanwhile inside that InitApp() thing I need to do this:

int width = GetSystemMetrics(SM_CXSCREEN);
int height = GetSystemMetrics(SM_CYSCREEN);

Which gets the desktop res for me to create a window, and show it to the world:

gWnd = CreateMainWindow("Production Line", gInstance, IDI_ICON1, IDI_ICON1, WindowProc, width, height, bwindowed, bborderless, 0, 0);
UpdateWindow(gWnd);
ShowWindow(gWnd,1); 

After that I initialise all the directx stuff, which I have written about a decade ago, its pretty easy to grab all that from the directxSDK. BTW that CreateMainWindow function is mine too, and looks like this:

HWND CreateMainWindow(char* appname,HINSTANCE hInstance,int IDI_SMALL_ICON,int IDI_TINY_ICON,
                       WNDPROC proc,int width,int height,bool bwindowed,bool borderless,int left,int top)
 {
     WNDCLASSEX wcex;
HICON icon = NULL;
HICON iconsmall = NULL;
if(IDI_SMALL_ICON == NULL)
{
    icon = (HICON)LoadImage(NULL,"data/icon.ico",IMAGE_ICON,0,0,LR_LOADFROMFILE);
}
else
{
    icon = (HICON)LoadImage(GetModuleHandle(NULL), MAKEINTRESOURCE(IDI_SMALL_ICON), IMAGE_ICON, 64, 64, 0);
    iconsmall = (HICON)LoadImage(GetModuleHandle(NULL), MAKEINTRESOURCE(IDI_TINY_ICON), IMAGE_ICON, 32, 32, 0);
}

wcex.cbSize           =    sizeof(WNDCLASSEX);
wcex.hInstance        =    hInstance;
wcex.lpszClassName    =    appname;
wcex.lpfnWndProc      =    proc;
wcex.style            =    CS_CLASSDC | CS_DBLCLKS;
wcex.hIcon            =    icon;
wcex.hIconSm          =    iconsmall;
wcex.hCursor          =    NULL;
wcex.lpszMenuName     =    NULL;
wcex.cbClsExtra       =    0 ;
wcex.cbWndExtra       =    0 ;
wcex.hbrBackground    =    (HBRUSH) GetStockObject (BLACK_BRUSH);

RegisterClassEx(&wcex); 

int flags = 0;
if(bwindowed)
{
    if(borderless)
    {
        flags  = WS_POPUP;
    }
    else
    {
        flags = WS_OVERLAPPED |  WS_CAPTION | WS_SYSMENU | WS_MINIMIZEBOX | WS_BORDER;
    }
}
else
{
    flags  = WS_POPUP;
}

HWND gWnd = CreateWindow(appname,appname,flags,
    left,top, width, height,
    NULL,//GetDesktopWindow(),
    NULL, hInstance, NULL);

return gWnd;
}

TBH the only fiddly bit is finding a program to actually create a decent windows ico .ico file in all of its myriad different sizes. Plenty of them exist, but its something you probably did not realise you need. I also have to use a batch file and a little command line thing to stuff the finished icon into the exe, so that it shows up in windows explorer as well as when the window is launched (weirdly these are different things…).

Thats pretty much all of the ‘initialisation’ code you need to create a windows app. Obviously you then need an actual graphics engine, but thats not *too bad* to write yourself if you are not coding something that is huge and requires amazing performance or tons of shaders.

In 2D, everything is pretty simple. You just need a sprite class (I have a Base Sprite, which is just 4 vertexes and a way to manipulate them by scaling, changing UV values, positioning them etc), maybe a textured sprite (the same but with a directx/opengl texture pointer), and if you want to optimise stuff you need a system that handles vertex buffers.

In general I just have one (big) vertex buffer that I stuff with sprites during the main game render loop, and then call draw() on it every time I need to change the texture or one of the render states.

Input stuff is pretty simple. You should just hook into the windows WM_LBUTTONDOWN messages and others like WM_CHAR, and do some processing on them. Way simpler than using middleware for all that. If you need to know the current state of a key you can use this:

bool CBaseInputManager::KeyDown(int key)
{
bool down = (bool)(GetAsyncKeyState(key)& 0xFF00);
return down;
}

I *do* use middleware for sound, but you can get pretty affordable, pretty simple sound middleware from lots of places these days. I have not updated my sound middleware for a LONG time. I don’t do any fancy sound processing so why would I? Playing an ogg file or streaming one…is not complex.

Now obviously if you have never coded outside unity, then there is a LOT of stuff you take for granted that you would have to now write some code for, but you only need to learn these things once. Loading in a file, or browsing a folder for files, is just a few lines of code. Even writing an absolutely bullet proof ini file loader that operates efficiently and correctly and without bugs is only 913 lines of code by my count, and thats a LOT of whitespace, helper functions and wrappers around it.

If you start writing an engine early, when you are still doing hobby games, preferably simple ones, you will find it quite easy to scale it up as you make more complex games. The code samples in this blog post have been stripped of their error checking, and some game-specific checks and extra processing to make them clear on first reading.

Basically every game I make involves me adding some new functionality to my engine. Production Line was the first game where I had to create an isometric object class, and isometric renderer (basically just a system to sort by Z and then render…), an animation compression system and some metric reporting and user-survey stuff, but that was probably just adding an extra 3-4% to the size of the total engine.

I wrote the multi-threading stuff ages ago (Democracy 3 if I recall), although I seriously improved it this time. The original windowed GUI system was coded for Starship Tycoon (OMG). Most of the directx stuff comes from Gratuitous Space Battles 2. The current text renderer dates back to Democracy 2 for its initial version. The vertex buffer code started in GSB 1.

Don’t get me wrong, there is a LOT of code in an engine, and it DOES take a fair bit of time, but its a big, long term investment that definitely pays dividends. I don’t have to worry if unity supports X or Y, or if it conflicts with Z, or if they are going to remove it without warning next week (or break it). All the code works in a style I like, and with absolutely zero bloat.

The source folder for my engine is 100 files and 620k total. Its not *that* big. Obviously it compiles pretty quickly and easily.

…and don’t forget you can still code using an IDE (Like Visual Studio) with built in code syntax highlighting and intellisense (I recommend visual assist!). Do not fall into the trap of thinking its unity OR just typing in a command line window without any help!

Fun with 64 bit programming

So my recent adventures in the land of code have taken me to port Production Line to 64 bit. The current 32bit build only allows me to access 2 GB of RAM and although even super large factories can fit in 800MB now, when you really pack things in and put the hours in, it *is* possible to hit 2 GB. With modding & any possible post-release expansion possibilities, there is arguable a need to remove that limit and so here we are.

I’ve basically done 2 days(ish) work on it, and have a release build and debug build 64 bit version of the game that seems to run just fine. It was relatively painless. The 3rd party stuff I use is mostly, Steam, some sound middle ware, Directx and an intel profiler, and all of this has 64 bit support, so the majority of the work has been going through the config for the game and changing include folders and paths to point to 64 bit DLLs and Lib files.

This has been complicated a bit by the mess that the Visual Studio (2013) software makes out of configurations. I can have a debug64 project config that then has the platform set to 32bit, and then god-knows where the exe gets put or which files get compiled and oh-my-god how messy. I think I have finally got close to getting it straight in my head, although I have ended up hard coding exe names and paths and may have to even rename my engines lib file to engine_64.lib to make ABSOLUTELY sure that it is not using the wrong lib file and thus mix-and matching.

I can totally see why people do not want to support both 32 bit and 64 bit versions of the same game, especially given the fact that, unbeknown to me, the mere *existence* of a 32 bit dll in the exe folder will seemingly stop my 64bit .exe launching. I guess you end up with separate folders? What a pain.

In terms of code, it was almost entirely painless. As I suspected, my one pain point was my GUI code for buttons. I have a base button type that takes a function pointer called BUTTONFUNC to execute when clicked. I cannot remember far enough back to work out why, but generally I end up passing an objects ‘this’ pointer as an (int) to the constructor for a button, if I want the button function to access it as data. So I end up writing code like this:

PCheckFree = new GUI_CheckButton(TRANSLATE(“CHOOSE_MISSION_FREEPLAY”), Freeplay, (int)this);

And then in the code that receives it I’d do this:

void GUI_Scenario::Freeplay(int data)
{
GUI_Scenario* pwin = (GUI_Scenario*)data;
pwin->SetType(SIM_Scenario::FREEPLAY);
}

Which is perfectly fine and lovely, assuming pointers are 32 bit and an int is 32bit. However, it turns out that porting to 64 bit is as simple as just replacing both users of (int) with (size_t) which varies based on platform, and voila! problem sorted. I expected this to be the first step in a whole world of nightmares,, but although I have not done serious testing yet, it appears to launch, run and allow me to load in massive save games, so I reckon I’m 95% of the way there. All I need to do know is investigate how all the various stores (Humble, Kartridge, Steam, Gog) handle multiple versions (64 bit versus 32 bit), to ensure I’m not leaving the tiny 32bit minority behind. I guess eventually that will not be an issue.

I’m definitely happy that this seems to have gone smoothly, as it amounts to days of coding and admin and investigation which are essential, but doesn’t make the game noticeably better for players, which is always a worry when the game is still in Early Access. Fun fun fun…

 

 

 

Optimizing Production Line app startup

Yeah I know its not long anyway…but its 2018 and on a CPU this speed, why is it not virtually instant?

I just multithreaded the steam app_init so that stuff is effectively ‘free’ now, and its time to see where the rest of the startup time is going. My current measurements from AQTime:

Drilling into this it looks like the easy wins will be inside the Main Menu constructor, as the init3d stuff probably cannot be multithreaded, and involves a lot of disk-bound stuff while I load textures, which is really tough to speed up (I could turn on my pak file code, but I’ll probably only do that just before final release).

Sadly almost all of the main menu constructor code looks like its the loading in of a big png file for my menu background. This is a 2560×1440 png file that is 7.5MB on disk. This is big, but I load in WAY more graphics than that when I do the pre-load textures on all the cars, which are in dds format. I’ll experiment with shifting it to a DDS file. This *should* be way faster, as a png has to be converted whereas a DDS file effectively *is* in the memory format used internally by directx, so its just a straight dump into memory…

That *dopes* actually drop it down to 17.2% (from 21.73%). I suspect there is a further optimisation in that this is currently not a power of two texture, so changing it to be one (as a test) yields…

OMG. its now 1.54%. Resizing that texture seems to take insane amounts of time, and this change knocks 0.5 seconds off my startup time. The next candidate is my sound engine. Multithreading its startup code completely takes it out of the picture too.

MainMenu initialise goes from 116ms to 4ms just by converting a non pow-2 png to a pow2 dds file, and now things look like this:

Total startup time has shrunk from 2.5 seconds to 1.9 seconds. Nothing anybody will consciously notice, but it makes me happier :D. Plus I *do* think that subconsciously people do feel the difference. Snappy start-up times are great for games when you want to have a quick blast, and the perceived responsiveness is bound to make people feel happier about their experience with the game. Plus it means less CPU draw and less battery drain on laptops.

This is for my car factory simulation game Production Line, currently in early access.

 

 

 

Programming video: VTune for thread concurrency

Not of interest to many, but I made a little demonstration video (using Production Line) to show how I can check out the concurrency of threads in the game using Intels VTune software. Enjoy:

Slow file access

If you read my blog often you will know I can be very irritated by poor performance in my code, or for that matter, anybodies. Firefox is possibly the most memory-wasting application in the known universe. Quite why it needs >3% of my entire CPU right now for me to just type these characters is beyond me. Despite this, such performance is not ‘noticeable’ in the same way that a sluggish GUI can be. When you click a button in a game, the resulting action needs to happen IMMEDIATELY for you to feel like you are using the interface, not fighting it. Thus when launching a dialog box in a game, the aim is always to have its initial loading time to be as minimal as possible. often thats easy, sometimes…not.

When you click the ‘load’ button on the main menu of Production Line (my latest game), it loads a dialog with a list (scrollable) of windows for each save game on the disk. There are thumbnails for each one showing the screen grab from when they were saved, plus some data about each save game. Example:

 

This probably sounds like it should be pretty fast to create, but actually its annoyingly, painfully slow. Before you ask, yes I do the initialisation ‘lazily’ in that I am not loading in textures for the save games until I draw them, so the ones that are currently not visible due to the scroll position have not slowed me down. Actually the slowdown is much simpler than that.

There are currently 25 savegames in my list, in a folder with 50 files (a thumbnail for each one is in the folder too). The files range from 600k to 176MB for the actual save games (XML format) and the thumbnails are tiny 50k jpgs. Why so slow?

At the very least I need to query data about 25 files here. The dialog box puts them in order of creation, and to ensure its really accurate, I dont use windows file attributes but actually crack open the XML to take a look at the header data inside. At this point, I extract the date, and time, and do a version check to reject super-old unusable saves. I strongly suspect that the delay I sometimes experience (only when I’ve been doing other stuff, and the files are not in the cache of the hard drive, or in windows RAM already) is actually not even the reading of the files, or the enumerating of them (50 is not many) but the accessing of them.

When you access a file in windows quite a lot of behind the scenes crap happens. Drives may have to be spun up (or not, depending on tech), maybe even network shares may have to be connected to (not in this case), maybe wireless network drivers need kicking out of sleep. Windows needs to check that you have permission to access that file, to compare the desired access against permitted access. It needs to navigate a chain of block links if the file is fragmented on disk, and as it does all of this, the users anti virus program will kick into gear, scanning the file (maybe even the entire thing, like my big 176MB xml?) for malware.

All of this takes TIME.

The worst thing is, this stuff all happens for each individual file, which is why game engines tend to use pak files. (I have support for them in my engine, just not using it yet). The problem is, users save games are one area where you likely really cannot use them. These are files created by the user, and its often helpful (especially during beta) for them to be simple files the users can access, delete if necessary, copy if necessary, email to the dev if necessary. So pak-filing them is not an option. There are many hacks I can think of, including maintaining a summary of the games in a single file I can update lazily at another time, but nothing that doesn’t generate more complexity and potential for bugs.

One solution, if I was really bored and desperate for speed, would be to embed the jpg into the xml, so that the umber of files instantly halved. Certainly a future option. I could also swap to compressed save games that were likely 1/10th (or less) the size, which would make debugging them a tad harder, but would mean much less raw data for windows and file-scanners to deal with.

I’m definitely not happy with this tiny, tiny (under half second) delay when you click that button :D