Game Design, Programming and running a one-man games business…

Category: programming

Optimizing Production Line app startup

Yeah I know its not long anyway…but its 2018 and on a CPU this speed, why is it not virtually instant?

I just multithreaded the steam app_init so that stuff is effectively ‘free’ now, and its time to see where the rest of the startup time is going. My current measurements from AQTime:

Drilling into this it looks like the easy wins will be inside the Main Menu constructor, as the init3d stuff probably cannot be multithreaded, and involves a lot of disk-bound stuff while I load textures, which is really tough to speed up (I could turn on my pak file code, but I’ll probably only do that just before final release).

Sadly almost all of the main menu constructor code looks like its the loading in of a big png file for my menu background. This is a 2560×1440 png file that is 7.5MB on disk. This is big, but I load in WAY more graphics than that when I do the pre-load textures on all the cars, which are in dds format. I’ll experiment with shifting it to a DDS file. This *should* be way faster, as a png has to be converted whereas a DDS file effectively *is* in the memory format used internally by directx, so its just a straight dump into memory…

That *dopes* actually drop it down to 17.2% (from 21.73%). I suspect there is a further optimisation in that this is currently not a power of two texture, so changing it to be one (as a test) yields…

OMG. its now 1.54%. Resizing that texture seems to take insane amounts of time, and this change knocks 0.5 seconds off my startup time. The next candidate is my sound engine. Multithreading its startup code completely takes it out of the picture too.

MainMenu initialise goes from 116ms to 4ms just by converting a non pow-2 png to a pow2 dds file, and now things look like this:

Total startup time has shrunk from 2.5 seconds to 1.9 seconds. Nothing anybody will consciously notice, but it makes me happier :D. Plus I *do* think that subconsciously people do feel the difference. Snappy start-up times are great for games when you want to have a quick blast, and the perceived responsiveness is bound to make people feel happier about their experience with the game. Plus it means less CPU draw and less battery drain on laptops.

This is for my car factory simulation game Production Line, currently in early access.

 

 

 

Slow file access

If you read my blog often you will know I can be very irritated by poor performance in my code, or for that matter, anybodies. Firefox is possibly the most memory-wasting application in the known universe. Quite why it needs >3% of my entire CPU right now for me to just type these characters is beyond me. Despite this, such performance is not ‘noticeable’ in the same way that a sluggish GUI can be. When you click a button in a game, the resulting action needs to happen IMMEDIATELY for you to feel like you are using the interface, not fighting it. Thus when launching a dialog box in a game, the aim is always to have its initial loading time to be as minimal as possible. often thats easy, sometimes…not.

When you click the ‘load’ button on the main menu of Production Line (my latest game), it loads a dialog with a list (scrollable) of windows for each save game on the disk. There are thumbnails for each one showing the screen grab from when they were saved, plus some data about each save game. Example:

 

This probably sounds like it should be pretty fast to create, but actually its annoyingly, painfully slow. Before you ask, yes I do the initialisation ‘lazily’ in that I am not loading in textures for the save games until I draw them, so the ones that are currently not visible due to the scroll position have not slowed me down. Actually the slowdown is much simpler than that.

There are currently 25 savegames in my list, in a folder with 50 files (a thumbnail for each one is in the folder too). The files range from 600k to 176MB for the actual save games (XML format) and the thumbnails are tiny 50k jpgs. Why so slow?

At the very least I need to query data about 25 files here. The dialog box puts them in order of creation, and to ensure its really accurate, I dont use windows file attributes but actually crack open the XML to take a look at the header data inside. At this point, I extract the date, and time, and do a version check to reject super-old unusable saves. I strongly suspect that the delay I sometimes experience (only when I’ve been doing other stuff, and the files are not in the cache of the hard drive, or in windows RAM already) is actually not even the reading of the files, or the enumerating of them (50 is not many) but the accessing of them.

When you access a file in windows quite a lot of behind the scenes crap happens. Drives may have to be spun up (or not, depending on tech), maybe even network shares may have to be connected to (not in this case), maybe wireless network drivers need kicking out of sleep. Windows needs to check that you have permission to access that file, to compare the desired access against permitted access. It needs to navigate a chain of block links if the file is fragmented on disk, and as it does all of this, the users anti virus program will kick into gear, scanning the file (maybe even the entire thing, like my big 176MB xml?) for malware.

All of this takes TIME.

The worst thing is, this stuff all happens for each individual file, which is why game engines tend to use pak files. (I have support for them in my engine, just not using it yet). The problem is, users save games are one area where you likely really cannot use them. These are files created by the user, and its often helpful (especially during beta) for them to be simple files the users can access, delete if necessary, copy if necessary, email to the dev if necessary. So pak-filing them is not an option. There are many hacks I can think of, including maintaining a summary of the games in a single file I can update lazily at another time, but nothing that doesn’t generate more complexity and potential for bugs.

One solution, if I was really bored and desperate for speed, would be to embed the jpg into the xml, so that the umber of files instantly halved. Certainly a future option. I could also swap to compressed save games that were likely 1/10th (or less) the size, which would make debugging them a tad harder, but would mean much less raw data for windows and file-scanners to deal with.

I’m definitely not happy with this tiny, tiny (under half second) delay when you click that button :D

An impossible bug. ARGGGH

Check out this code from production line for displaying pie charts of expenses.  I declare arrays of float totals for each of 3 pie charts.

 float totals[NUMPIES];
 totals[PIE1] = 0;
 totals[PIE24] = 0;
 totals[PIEALL] = 0;

 

Then I declare a 2-dimensional array of floats which I fill with some data. As I build up those amounts I also update the totals:

float amounts[NUM_FINANCE_CATEGORIES][NUMPIES];

for (int r = 0; r < NUM_FINANCE_CATEGORIES; r++)
 {
 if (r != FC_CAR_SALES)
 {
 amounts[r][PIE1] = SIM_GetFinanceRecords()->GetAmount(1, (FINANCE_CATEGORY)r);
 amounts[r][PIE24] = SIM_GetFinanceRecords()->GetAmount(24, (FINANCE_CATEGORY)r);
 amounts[r][PIEALL] = SIM_GetFinanceRecords()->GetAmount(-1, (FINANCE_CATEGORY)r);

totals[PIE1] += amounts[r][PIE1];
 totals[PIE24] += amounts[r][PIE24];
 totals[PIEALL] += amounts[r][PIEALL];
 }
 }

 

Then some simple resetting of data, which is irrelevant for this bug, and a check that I am not about to divide by zero:

Pies[PIE1]->Clear();
 Pies[PIE24]->Clear();
 Pies[PIEALL]->Clear();

if (totals[PIE1] <= 0 || totals[PIE24] <= 0 || totals[PIEALL] <= 0)
 {
 return;
 }

 

Then the final code:

 //now create
 for (int r = 0; r < NUM_FINANCE_CATEGORIES; r++)
 {
 if (r != FC_CAR_SALES)
 {
 for (int p = 0; p < NUMPIES; p++)
 {
 float perc = amounts[r][p] / totals[p];
 assert (perc >= 0 && perc <= 1.0f)

Hold ON STOP!. How can that assert ever trigger? EVER? (it does for some people). Its driving me mad :D. It can ONLY trigger if one of the amounts in the array is less than 0% or more than 100% of the total. I KNOW that the total is greater than zero, so the amount must be greater than zero. The total is only comprised of the sum of the amounts. There is no way these numbers can be out of synch, PLUS, they are all floating point vars so there is no rounding going on… or is it maybe a tiny tiny quantizing thing? (I’ve update the game with extra debug data for this error so I’ll find out soon). Surely thats the only explanation, and perc is something like 1.000000001%?

 

 

Coding your own ‘middleware’ is easier than you think

I have a system in the early access version of production Line that when an error happens that I assert() on, it logs that error out to a debug file, along with a timestamp, and the file name and line number of code where the assert failed. This is easy to do (google it). I recently changed this code so it also posts the error message, line number and filename data to my server (anonymously, I have no idea which player triggered it). That code basically just uses the WININET API to silently post some URL variables to a specific php page. That page then validates the data to make it safe, and runs an SQL query to stick the data into a database on my server.

IU can run SQL queries on the server to see what bugs have happened in the last 24 hours, which ones happened the most, what the version number of the game was, and how often they are triggering etc.

This has proven to be awesome.

The default solution these days to this kind of stuff is to buy into some middleware to do this, but thats not how I roll. I also have some balance-tracking stats analysis stuff that does the same thing. I can tell if the game is too hard, or too easy, or if players dont get to build electric cars until hour 300 in the game…and all kinds of cool stuff. This is the sort of thing that middleware companies with 20 staff, a sales team and a slick marketing campaign try to flog to you at GDC in the expo. Its really not that hard, it really is not rocket science. I get the impression that the majority of coders think that writing this sort of stuff is an order of magnitude harder than it actually is.

Oh also, production Line checks once per day on start-up to see if there is a newer version, then pops up a notice and a changelist if there is (for non steam versions). Thats home-grown code too, and its easy, EASY to do,m once you actually sit down and work out whats involved.

I know the standard arguments in favor middleware ‘it comes with tech support, its faster than coding it yourself, it allows you to leverage the experience and knowledge of others’, but I reject all of them. The only reason I know how to use WinInet, php and SQL and all that sort of stuff is because I taught myself. I taught myself ONCE in the last 36 years of programming, and I know how to do it now., I have my old code, and my own experience and skills. I dont care if XYZ Middleware INC is going to go bust, stop supporting my platform, double their prices, or stop answering emails, I have all this experience in house.

When you look at most middleware, its vastly bigger, more feature packed and complicated than you need it to be. Middleware has to be, because it has to be all things to all people. My stats tracking and error-reporting doesn’t have to work on IOS or on OSX or on mobile, or XBox or Playstation. it doesn’t have to be flexible enough to talk to four different database types, or support Amazons AWS. I don’t NEED any of that stuff, so guess what… the code to do what I *actually* need to do, is incredibly, incredibly simple. Don’t think about middleware contracts and APIs and excuses not to write the code, ask yourself ‘what exactly does the code need to do here anyway?’

I think when coders do that you will find, its actually way easier than you think. I did my error reporting code, both the client side and the server side, and tested and debugged in in about an hour. This stuff is not complex. Stop buying ‘fully-featured’ bloatware you don’t need.