Game Design, Programming and running a one-man games business…

Month: June 2019

Starting the game: An in depth profiling look

How long does your indie game take to start up? from clicking the icon to actually being able to take input at the main menu? Just for fun, I decided to analyze whats involved in doing so for mine.

Because the aim here is to actually analyze the REAL impact, not the best case, I need to ensure that the game (Production Line) is not just happily sat there all in RAM from a recent run-through, so it seems best to oh…maybe launch battlefield V beforehand (and quit it) just to populate disk/RAM with a load of other stuff and do my best to evict all my games code.

Then…its time to fire-up aqtime and take a look. I decided to do line-level, rather than just function-level analysis, which slows the game massively, taking 17 seconds to start (the reality is dramatically faster), but I can still do relative comparisons.

First thing to notice is that pretty much the entire time is inside Game::InitApp() which makes sense.

Rather worryingly though, the vast majority appears to be inside SIM_Threadmanager::Initialise. That *may* be an artifact of aqtimes approach to thread profiling, but worth taking a look inside anyway… And it turns out that 100% of that time is inside SetThreadName() (which i only need for debugging anyway). This is a rare bit of code that I don’t understand well, and was from the evil interwebs:

#pragma pack(push,8)
typedef struct tagTHREADNAME_INFO
{
	DWORD dwType; // Must be 0x1000.
	LPCSTR szName; // Pointer to name (in user addr space).
	DWORD dwThreadID; // Thread ID (-1=caller thread).
	DWORD dwFlags; // Reserved for future use, must be zero.
} THREADNAME_INFO;
#pragma pack(pop)

void SetThreadName(DWORD dwThreadID, char* threadName)
{
	THREADNAME_INFO info;
	info.dwType = 0x1000;
	info.szName = threadName;
	info.dwThreadID = dwThreadID;
	info.dwFlags = 0;

	__try
	{
		RaiseException(MS_VC_EXCEPTION, 0, sizeof(info) / sizeof(ULONG_PTR), 
(ULONG_PTR*)&info);
	}
	__except (EXCEPTION_EXECUTE_HANDLER)
	{
		volatile int foo = 9;
	}
}

The exception is basically ALL of the time. WTF? Apparently there is a less hacky way outlined here: https://docs.microsoft.com/en-us/visualstudio/debugger/how-to-set-a-thread-name-in-native-code?view=vs-2019 Which I will try later. I suspect the waiting for visual studios debugger is the cause of the problem.

Anyway…onwards and upwards, so whats next? It basically all Init3D() (click to enlarge)

So basically my DirectX initialisation and the shadermanager stuff is most of the problem. I suspect the DirectX initialisdation may be too black-boxed for me to influence further. The first big chunk is this line:

PD3D9 = Direct3DCreate9(D3D_SDK_VERSION);

Which takes up 34.74% of the start time. The next slow bit is the largest at 41% which is:

hr = PD3D9->CreateDevice( AdapterToUse, DeviceType, WindowHandle,
 D3DCREATE_SOFTWARE_VERTEXPROCESSING, &PresentParameters, &PDevice);

So…holy crap. how can that line of code even be run? This can only happen if my checkcaps() code suggest the video card does not support hardware transform and lighting. I suspect some of the reporting here must be nonsense? Especially as my own debug logs suggest that the hardware TNL version is the one than ran… FFS :( lets look outside that code then…

Most of the slowdown is in shader manager, which loads 11 shaders:

so it looks like about half the loading time here is actually spent writing out debug data! This is hard to avoid though, as I do find this data invaluable for detecting errors. And because an app can crash and lose all its data, I flush each line of my debug logs to disk with a hard drive flush on each line…

…so interestingly all the time seems to be inside OutputDebugString, which is only of any real use when the debugger is running. However! I *do* want to see that data in both release builds, and debug builds. Maybe I need a flag to tell if a debugger is present when the debug engine starts up? As a first pass I should at least build up a char* with the newline in to avoid twice the OutputDebugString calls. Here is the new code and timings.

Ooooh. I’ve halved the time of it. I’ve done the same with my non-directx debug code too. Now I’ll try changing that thread stuff… It turns out that SetThreadDescription is windows 10 only, so I need a different system (and apparently would need to update my platform SDK…urrrgh), so maybe best to just skip calling that code when no debugger is detected?

This works (using isDebuggerPresent) but the profiler actually trips that flag too, so to set it work I needed to compare time stamps on debug files. Without the debugger, it looks like time from app start to menu ready is 0.395 seconds. With the debugger its… 0.532 seconds.

That sounds pretty small, but actually I’m quite happy as I lost ZERO functionality, and the changes to the debug data will affect any time that data is written, not just during startup. 9Its not a lot, but there is *some*, and I’m an efficiency obsessive.

I think I’ll put a clause around the debugengines OutputDebugString and nuke that unless IsDebuggerPresent() too :D

The curse of staggeringly slow code

No, I’m not talking about mine, but about *other peoples code* that I encounter on a day to day basis. Some choice examples:

When I start aqtime ( a profiling app, ironically), it hangs for about 10 seconds. then when I load a project (the file is under 100k) it hangs for another ten seconds.

There is no discernible network activity during this time, and the CPU is not thrashed either. How is this even POSSIBLE? A quick check shows that my i7 3600 can do 106 trillion instructions per second. (106,000 MIPS). Thats insane. It also means that in this ten seconds, it can do one thousand trillion instructions. To do seemingly…nothing

Also… for my sins I own a Samsung smart TV. When I start up that pile of crap, if often will not respond to remote buttons for about eight seconds, and even then, it queues them up, and can lag processing an instruction by two or three seconds. This TV has all eco options disabled (much though that pains me), and has all options regarding uploading viewing data disabled. Lets assume its CPU runs at a mere 1% of the speed of my i7, that means it has to get buy on a mere 10 trillion operations per second. My god, no wonder its slow, as I’m quite sure it takes a billion instructions just to change channels right? (even if it did, it could respond in 1/10,000th of a second)

These smiling fools would be less chirpy if they knew how badly coded the interface was

I just launched HTMLTools, some software I use to edit web pages, and it took 15 seconds to load up an present an empty document. fifteen seconds, on an i7 doing absolutely nothing of any consequence.

Why the hell do we tolerate this mess? why have we allowed coders to get away with producing such awful, horrible, bloated work that doesn’t even come close to running at 1% of its potential speed and efficiency.

In any other realm this would be a source of huge anger, embarrassment and public shaming. My car can theoretically do about 150mph. imagine buying such a car and then realizing that due to bloated software, it can only actually manage 0.1 miles per hour. Imagine turning on a 2,000watt Oven, and only getting 2 of those watts used to actually heat something. We would go absolutely bonkers.

an intel i7. So much capability, so little proper usage.

There is a VAST discrepancy between the amount of time it takes optimized computer code to do a thing, and the amount of time the average consumer/player/citizen thinks it will take. We need to educate people that when they launch an app and it is not 100% super-responsive, that this is because it is badly, shoddily, terribly made.

Who can we blame? well maybe the people who make operating systems for one, as they are often bloated beyond belief. I use my smartphone a fair bit but not THAT much, and when it gets an update and tells me its ‘optimizing’ (yeah right) 237 apps… I ask myself what the hell is all of this crap, because i’m sure I didn’t install it. When the O/S is already a bloated piece of tortoise-ware, how can we really expect app developers to do any better.

I think a better source of blame is people who write ‘learn C++ in 7 days’ style books, who peddle this false bullshit that you can master something as complex as a computer language in less time than it takes to binge watch a TV series. Even worse is the whole raft of middleware which is pushed onto people who think nothing of plugging in some code that does god-knows-what, simply to avoid writing a few dozen lines of code themselves.

We need to push back against this stuff. We need to take a bottom-up approach where we start with what our apps/operating systems/appliances really NEED to do, and then try to create the fastest possible environment for this to happen. The only recent example of seen of this is the designing of a dedicated self-driving computer chip for tesla cars. very long explanation below: (chip stats about 7min 30 in)

Why I am obsessed with electric cars (esp: Tesla)

If you follow me on twitter you will know I bang on about electric cars a lot, specifically Teslas, and why I get angry at the FUD and nonsense spread about them online. Why do I care?

Lets get the disclaimers out of the way. Yup, I own one (A 2015 85D Tesla model S with autopilot 1) and yup, I own some tesla stock. Obviously I am biased because I don’t want people to criticize my purchase choice (I am human) and because I have some financial interest (although TBH thats a relatively minor concern. I have stock in Microsoft and Nvidia too but I don’t bang on about em…). So given those obvious points, why else do I care?

Climate Change

First up..the obvious one. Climate change is real. Its also predominantly caused by humans, specifically CO2 emissions, and if you actually ‘do not believe’ that, then please think about what other widely accepted scientific conclusions you would like to dispute. Maybe you don’t believe in magnetic forces either? or perhaps even gravity? maybe the earth is not a sphere but flat? There is a wealth of scientific consensus on this, and you should not only accept the fact, but be absolutely terrified of the consequences. Don’t think about ‘it getting a bit warmer’ think about agricultural yield collapse, food price spikes, food-rioting, mass immigration, resource-wars and global upheaval. This should scare the crap out of you…

…and one of the things we as individuals can do is switch to a cleaner form of transport, notably: an electric car. They are not practical for everyone right now, but will be very soon, and once you buy one you dramatically cut the amount of CO2 you personally are stuffing into the atmosphere. Its a great way to do your bit.

Pollution

Secondly…Pollution. EVs not only emit no CO2 at the tailpipe (and electricity grids get greener every day), but also zero fumes or pollutants of any kind. That means cleaner cities, quieter roads (less noise pollution!) and fewer kids with asthma. And the car behind you on the road is no longer sucking up your exhaust fumes and blowing them into the driver & passengers faces. Plus the newer teslas have HEPA filters in that mean the air in the car will be substantially cleaner than the air outside. ideal for polluted cities.

People sometimes repeat some FUD about cobalt, implying its all from congo, and batteries are full of it. Actually its a mere 3% now, and dropping to 0% soon. Plus its dramatically less of a problem than the impacts of oil dependence…

Convenience

This is the one people just do not get, and will NOT accept…until they own an EV. The caveat here is assuming you have off-road parking at home, OR you work somewhere that has an EV charging point in the car park. (This is getting much, much more common).

Charging an EV is super-cheap (here in the UK I work it out to be just under £0.04 per mile in ‘fuel’, assuming 100% home charging), and actually MORE convenient than owning a petrol/diesel car. The fact is, with an EV, you have a fuel station *at your house*, and it can fill up while you sleep. In some cases you can set the car to charge during off-peak (cheaper) electricity times! The fact that the car charges while you sleep means every day when you get in the car, it has a full battery, so you can drive maybe 200-240 miles before recharging (real-world range in UK).

That 240 sounds low compared to your petrol car, until you realize that petrol car has to go to a special recharging place to get fuel (which costs a fortune), and where you have to stand there like an idiot holding a trigger to fill it up. Oh BTW that fuel is smelly, environmentally damaging and catastrophically dangerous.

On the average day, you do NOT drive 240 miles, and if you *do*, you are likely on a motorway, where you can stop and charge your car (still cheaper than petrol) while you grab a coffee and a donut. Charging speeds are getting faster than ever:

TBH, like 99% of tesla owners, even though I have access to the amazing supercharger network (which the car auto-navigates me to if it thinks I need power), I hardly use it, unless I drive to London and back with passengers in the car. Even then, I don’t *wait* for it, I just pick up an extra 50-60 miles while I have a coffee that I’d have stopped for anyway. No queuing to pay, no holding a pump, no logging in or barcode scanning, just stick the cable in and go grab coffee…

Technology

Electric cars have phenomenal batteries. These are NOT the same as the batteries in your mobile phone or laptop. They do not noticeably lose any charge (in fact the range of my car has gone UP since I bought it…thanks free software upgrades over the internet!). You do *not* have to give even the slightest thought to replacing your battery. You will likely need several new batteries for your old petrol car in the same time that you would notice even a minor degradation in your EV range over time. Battery tech in 2019 is amazing.

Performance

Holy crap they are fast. You might not care (I only care a bit), but since owning my EV, going back to my wife’s car (lexus CT200H hybrid) feels like driving a horse and cart, even in ‘sport mode’. EVs have instant power, real throw-you back in your seat with some force’ levels of power. For real car-geeks who want something sporty, trade in your petrol-car now, its history.

Maintenance

LOL. Whats that? I’ve owned my car 3 and a bit years now. It had an intermittent screen problem which is being fixed this week (under warranty), but apart from that and a minor thing with one door handle (see above, fixed), nothing has gone wrong. It was serviced once, but TBH it didn’t need it. Annual service? LOL. why? whats going to go wrong? the exhaust (nope)? radiator (no)? the drive shaft? (no)… EVs are actually WAY simpler than petrol cars. They are electric motors, a battery…errr and seats and doors I guess. The maintenance costs on them are *trivial*. Oh and satnav updates are streamed automatically while I sleep. For free, obviously.

The Future

Tech like EVs represent tipping-points. Right now they seem niche, but the sales are accelerating FAST, despite a super-well-funded FUD and bullshit campaign by multiple dying industries. Right now finding a petrol station for your old-tech car is easy, ditto a mechanic but that will crossover soon. The future of cars is undeniably electric, and we aren’t far off the point where the re-sale value of a petrol car starts to drop when people realize their next car will be electric, even if they won’t be able to buy one for a few years.

Safety

The top 3 safest cars on the planet right now? All EVs:

…oh and no engine in front of you means your body is not crushed in a front impact, plus the whole front of the car is a crumple zone. And the battery in the floor makes them bottom heavy, meaning they *do not roll*. Oh and I nearly forgot the complete lack of 50 liters of highly flammable liquid that just isn’t in an EV. You want a safe car? You want an EV.

Oh and BTW all electric cars are automatics. Learning to operate a clutch and a gear stick is so 1970s. This isn’t a skill we need to bother learning soon. I thought I’d hate an automatic but OMG I love it. Assing around with gears feels like being in some costume drama fiddling with cumbersome stupid old technology for a joke. I don’t miss it one bit.

I feel very strongly about electric cars mostly because there are so many lies spread about them. Mine is my favorite purchase of all time, except perhaps my house, and I’m not a car guy. I still don’t really know what torque is or what any bits of a traditional car do, nor do I care. But I like cutting edge tech that is so cool it makes you laugh out loud.

If you ever get an opportunity to try one, do so. Even the harshest skeptics are won over the minute they drive one.

BTW if you do buy a tesla, using this code ( http://ts.la/cliff7605 ) gets you some free supercharger miles. I already have unlimited for buying an early one, but at the time of writing this earns you 1,000 free miles.