Category: programming

So my recent adventures in the land of code have taken me to port Production Line to 64 bit. The current 32bit build only allows me to access 2 GB of RAM and although even super large factories can fit in 800MB now, when you really pack things in and put the hours in, it *is* possible to hit 2 GB. With modding & any possible post-release expansion possibilities, there is arguable a need to remove that limit and so here we are.

I’ve basically done 2 days(ish) work on it, and have a release build and debug build 64 bit version of the game that seems to run just fine. It was relatively painless. The 3rd party stuff I use is mostly, Steam, some sound middle ware, Directx and an intel profiler, and all of this has 64 bit support, so the majority of the work has been going through the config for the game and changing include folders and paths to point to 64 bit DLLs and Lib files.

This has been complicated a bit by the mess that the Visual Studio (2013) software makes out of configurations. I can have a debug64 project config that then has the platform set to 32bit, and then god-knows where the exe gets put or which files get compiled and oh-my-god how messy. I think I have finally got close to getting it straight in my head, although I have ended up hard coding exe names and paths and may have to even rename my engines lib file to engine_64.lib to make ABSOLUTELY sure that it is not using the wrong lib file and thus mix-and matching.

I can totally see why people do not want to support both 32 bit and 64 bit versions of the same game, especially given the fact that, unbeknown to me, the mere *existence* of a 32 bit dll in the exe folder will seemingly stop my 64bit .exe launching. I guess you end up with separate folders? What a pain.

In terms of code, it was almost entirely painless. As I suspected, my one pain point was my GUI code for buttons. I have a base button type that takes a function pointer called BUTTONFUNC to execute when clicked. I cannot remember far enough back to work out why, but generally I end up passing an objects ‘this’ pointer as an (int) to the constructor for a button, if I want the button function to access it as data. So I end up writing code like this:

PCheckFree = new GUI_CheckButton(TRANSLATE(“CHOOSE_MISSION_FREEPLAY”), Freeplay, (int)this);

And then in the code that receives it I’d do this:

void GUI_Scenario::Freeplay(int data)
{
GUI_Scenario* pwin = (GUI_Scenario*)data;
pwin->SetType(SIM_Scenario::FREEPLAY);
}

Which is perfectly fine and lovely, assuming pointers are 32 bit and an int is 32bit. However, it turns out that porting to 64 bit is as simple as just replacing both users of (int) with (size_t) which varies based on platform, and voila! problem sorted. I expected this to be the first step in a whole world of nightmares,, but although I have not done serious testing yet, it appears to launch, run and allow me to load in massive save games, so I reckon I’m 95% of the way there. All I need to do know is investigate how all the various stores (Humble, Kartridge, Steam, Gog) handle multiple versions (64 bit versus 32 bit), to ensure I’m not leaving the tiny 32bit minority behind. I guess eventually that will not be an issue.

I’m definitely happy that this seems to have gone smoothly, as it amounts to days of coding and admin and investigation which are essential, but doesn’t make the game noticeably better for players, which is always a worry when the game is still in Early Access. Fun fun fun…

Yeah I know its not long anyway…but its 2018 and on a CPU this speed, why is it not virtually instant?

I just multithreaded the steam app_init so that stuff is effectively ‘free’ now, and its time to see where the rest of the startup time is going. My current measurements from AQTime:

Drilling into this it looks like the easy wins will be inside the Main Menu constructor, as the init3d stuff probably cannot be multithreaded, and involves a lot of disk-bound stuff while I load textures, which is really tough to speed up (I could turn on my pak file code, but I’ll probably only do that just before final release).

Sadly almost all of the main menu constructor code looks like its the loading in of a big png file for my menu background. This is a 2560×1440 png file that is 7.5MB on disk. This is big, but I load in WAY more graphics than that when I do the pre-load textures on all the cars, which are in dds format. I’ll experiment with shifting it to a DDS file. This *should* be way faster, as a png has to be converted whereas a DDS file effectively *is* in the memory format used internally by directx, so its just a straight dump into memory…

That *dopes* actually drop it down to 17.2% (from 21.73%). I suspect there is a further optimisation in that this is currently not a power of two texture, so changing it to be one (as a test) yields…

OMG. its now 1.54%. Resizing that texture seems to take insane amounts of time, and this change knocks 0.5 seconds off my startup time. The next candidate is my sound engine. Multithreading its startup code completely takes it out of the picture too.

MainMenu initialise goes from 116ms to 4ms just by converting a non pow-2 png to a pow2 dds file, and now things look like this:

Total startup time has shrunk from 2.5 seconds to 1.9 seconds. Nothing anybody will consciously notice, but it makes me happier :D. Plus I *do* think that subconsciously people do feel the difference. Snappy start-up times are great for games when you want to have a quick blast, and the perceived responsiveness is bound to make people feel happier about their experience with the game. Plus it means less CPU draw and less battery drain on laptops.

This is for my car factory simulation game Production Line, currently in early access.

Not of interest to many, but I made a little demonstration video (using Production Line) to show how I can check out the concurrency of threads in the game using Intels VTune software. Enjoy:

If you read my blog often you will know I can be very irritated by poor performance in my code, or for that matter, anybodies. Firefox is possibly the most memory-wasting application in the known universe. Quite why it needs >3% of my entire CPU right now for me to just type these characters is beyond me. Despite this, such performance is not ‘noticeable’ in the same way that a sluggish GUI can be. When you click a button in a game, the resulting action needs to happen IMMEDIATELY for you to feel like you are using the interface, not fighting it. Thus when launching a dialog box in a game, the aim is always to have its initial loading time to be as minimal as possible. often thats easy, sometimes…not.

When you click the ‘load’ button on the main menu of Production Line (my latest game), it loads a dialog with a list (scrollable) of windows for each save game on the disk. There are thumbnails for each one showing the screen grab from when they were saved, plus some data about each save game. Example:

This probably sounds like it should be pretty fast to create, but actually its annoyingly, painfully slow. Before you ask, yes I do the initialisation ‘lazily’ in that I am not loading in textures for the save games until I draw them, so the ones that are currently not visible due to the scroll position have not slowed me down. Actually the slowdown is much simpler than that.

There are currently 25 savegames in my list, in a folder with 50 files (a thumbnail for each one is in the folder too). The files range from 600k to 176MB for the actual save games (XML format) and the thumbnails are tiny 50k jpgs. Why so slow?

At the very least I need to query data about 25 files here. The dialog box puts them in order of creation, and to ensure its really accurate, I dont use windows file attributes but actually crack open the XML to take a look at the header data inside. At this point, I extract the date, and time, and do a version check to reject super-old unusable saves. I strongly suspect that the delay I sometimes experience (only when I’ve been doing other stuff, and the files are not in the cache of the hard drive, or in windows RAM already) is actually not even the reading of the files, or the enumerating of them (50 is not many) but the accessing of them.

When you access a file in windows quite a lot of behind the scenes crap happens. Drives may have to be spun up (or not, depending on tech), maybe even network shares may have to be connected to (not in this case), maybe wireless network drivers need kicking out of sleep. Windows needs to check that you have permission to access that file, to compare the desired access against permitted access. It needs to navigate a chain of block links if the file is fragmented on disk, and as it does all of this, the users anti virus program will kick into gear, scanning the file (maybe even the entire thing, like my big 176MB xml?) for malware.

All of this takes TIME.

The worst thing is, this stuff all happens for each individual file, which is why game engines tend to use pak files. (I have support for them in my engine, just not using it yet). The problem is, users save games are one area where you likely really cannot use them. These are files created by the user, and its often helpful (especially during beta) for them to be simple files the users can access, delete if necessary, copy if necessary, email to the dev if necessary. So pak-filing them is not an option. There are many hacks I can think of, including maintaining a summary of the games in a single file I can update lazily at another time, but nothing that doesn’t generate more complexity and potential for bugs.

One solution, if I was really bored and desperate for speed, would be to embed the jpg into the xml, so that the umber of files instantly halved. Certainly a future option. I could also swap to compressed save games that were likely 1/10th (or less) the size, which would make debugging them a tad harder, but would mean much less raw data for windows and file-scanners to deal with.

I’m definitely not happy with this tiny, tiny (under half second) delay when you click that button :D