Game Design, Programming and running a one-man games business…

Month: November 2022

Inertia scaling in Democracy 4 (new feature)

I recently added some code to Democracy 4 which introduces a new configurable option for people who are really into the game. This is not ‘live’ yet but will be in the next update. I have not totally settled on a name for it yet, but I think I’ll call it something like ‘opinion inertia boost’, because that ties in with other uses of the term throughout the game. So what is it?

Inertia already exists in the game. Every effect that a policy, an event, a dilemma choice or anything else may have on anything can have some inertia applied to it. This means that instead of X affecting Y, and it being a simple equation, in fact its an equation that takes the average value of X over a certain number of turns, which is an integer called ‘inertia’. You can see the value of inertia at the far right of any of the indicators in the game that show an effect:

The inertia is a value set by me, or by a modder, and its fixed. Its basically part of the equation, and set in stone. There has not yet been any way to change this by the player or during the game.

Inertia plays a vital role in making the game both realistic and fun. In practice, if you introduce grants for business startups, it does not convert 500,000 citizens to become capitalists the next day. The impact is going to take years. people need to hear about the grants, apply for them, see their business do well, tell their friends, people have to read newspaper stories about this happening… and perhaps even more relevantly, people who were once socialists but get ‘converted’ to capitalism have to slowly get out of the habits of their previous views, and adopt new ones, even in opposition to their friends/family and peers.

I see this sort of thing happen quite a lot, and have witnessed it with both friends and relatives. When peoples circumstances change, they tend to continue to cling to their existing core beliefs and principles, but over a period of time, opinions will change. With weakly held opinions, its quick to change, but with deeply ingrained beliefs, especially those that come from family, change can take a very long time.

I’m an example of this myself. I grew up in a very working class circumstance. My mother was a trade union representative at work, where she worked as a filing clerk for a trade union HQ (different union!) so she was Trade Unionist to the power of two! My father was also a union rep. We went on exciting day trips to the labour party conference. I read the communist manifesto, and Das Kapital as a teenager.

Fast forward thirty five years and I am a director of two companies, who trades on the stock market, and has employed people and run a multinational company (albeit v small). I’ve even worked on stock market trading floors. I own a single book by Ayn Rand (which I thought was interesting, albeit pretty ranty and very repetitive). My views have definitely changed over time, and with changes in circumstances.

The point is… change of core beliefs takes time. We all think we would keep the same principles until death, but thats unlikely. Societal change is very real. You might think your views regarding being liberal/conservative or left/right are very deeply held sensible views you independently arrived at, but this is very unlikely. I thought I was right as a teenage communist.

So the big question is… how long does it take for politics and society to convert someone from being hard right to hard left, or liberal to conservative? I have no definitive answer, but its a long time. Democracy 4 limits all inertia to a maximum value of 32, for technical reasons, which is 8 years. probably not long enough…

Like all games, Democracy 4 has to balance fun with realism. Not many games have a core mechanic of it taking 32 turns for you to see the impact of your choices, especially not if you can lose the game in an election in maybe 16 turns. Thats pretty brutal. Not only does this make the game hard, it also makes it frustrating. If inertia is too high, many players will just think ‘the game is random’ and complain about the RNG. Ironically there are very very few random numbers in the game…

Plus many players will not bother reading the tutorial or looking at the tooltips. I bet a good 30% of the people who play the game don’t even know inertia is a thing, or where to find it. Most players don’t really know what they are doing, because people have 1,000 games and no time to devote to learning the intricacies. If I modelled inertia realistically, a lot of frustrated players would have no fun.

…and yet…

There are definitely players who feel that they would prefer realism over ‘simple fun’. These players are frustrated that real societal change feels so easy in the game. If they want to convert France to a Capitalist state, they WANT to have to really struggle to take its left leaning population slowly along for the ride. They WANT change to be slow, and take longer, which is why I am adding this cool little slider to the games option screen:

This slider defaults to zero, where it has no impact, and goes as high as to add a 3x modifier. At the far right it will triple the inertia values for all inputs to socialism and liberalism. This will make all policies and events and dilemma choices act more slowly on the membership of these voter groups. (One value affects membership for both socialism and capitalism, on a spectrum and the same for liberal/conservative).

I’ve written all the code, and tested it works on both new games and save games. Everything seems fine but I will do a bunch of playthroughs before I do a new update that includes this slider. I’m pretty sure that 95% of players will never experiment with it at all, but I have enough players that I think its worth adding it all the same. It definitely gives me an easy answer to anybody who comments at me telling me that this part of the game is unrealistic!

Note that this difficulty in changing the country’s views is very much a real world modern problem. Even if Joe Biden was a communist, there is no way he can convert the majority of the US electorate over to his views in a single term. Arguably all Obama managed to do, in his entire tenure, is to get the affordable health care act in place, and it comes nowhere close to being a state health service like the NHS in the UK. Even a popular US president, with control of the house & senate, has to move extremely slowly in changing what the acceptable size of the state is, or changing social policy.

Trump found it almost impossible to actually build a wall, Obama found it impossible to close Guantanamo. Blair did very little in terms of raising taxes, Thatcher did very little in lowering them. Change takes real time.

One final thing: This slider only affects liberal/conservative and socialist/capitalist. I am unsure whether it should also affect membership of the religious group. Opinions very much welcome. I don’t think it should affect most groups, as peoples membership of these is a lot more fickle and easily swayed.

I finally have a home battery!

At long last, its finally installed and actually working, and after a very long and very tedious process that I will not bore you with (much) my house now has a 9.5kwh home battery connected up and working in the cellar. Its actually powering everything in my house right now, and is super cool.

This has been a long process because just when I started to ask a company to install one, UK electricity prices went absolutely insane, and everyone and their dog suddenly wanted a home battery installation. Getting a Tesla powerwall would have meant possibly an even longer wait, and to be honest, they were quite pricey compared with what I eventually chose (a Givenergy 9.5kwh one), but even when I found an installer, and agreed to have a battery installed, there were endless delays. Initially I was going to have a 9.5kwh (latest model) battery, then it became obvious they were hugely delayed, so opted for a single 8.4kwh one, then at the very last minute it turned out a 9.5kwh one was available, and as this was the latest tech, that allowed 100% depth of discharge and unlimited cycles (basically you can fully fill/empty the battery whenever you like without affecting the warranty), so we went with that.

Getting people to come around and install the battery was one thing. Getting it actually finished was another. The installation has been a real pain, but TBH that seems to be mostly down to huge demand and chaos among all the companies doing this sort of thing right now. We had the battery installed, but dormant for about 4 weeks until a vital missing component showed up (a wifi dongle that allowed it to talk to my home network, and thus back home to the battery management system run by the battery company).

Now its all installed, it all feels like it sort of went ok though, because my ideal scenario was to put it in our cellar, which is damp, and dark and empty, and thus the perfect location. Why take up room in the house with a great big metal box when you are almost never going to have to physically go see it? I had worried that the installers would be negative about anything ‘non-standard’, but were happy to run the power cable from our kitchen fuse box outside the house, along a wall, then down through a hole into the cellar and the battery. I’m very happy with where it is:

When it was installed I asked them to set up a double size wooden board on the wall and allow space for a second similar size battery if I wanted to add another one at some stage. In practice, I doubt this will happen. The picture shows the battery (at the bottom) and above it is the inverter. Basically an inverter converts DC to AC or vice versa. This battery is ‘AC-coupled’ which means its on the ‘house’ side of the setup, and is wired in to the fusebox pretty much like anything else. It needs the inverter to convert that AC power so it can be stored in, or sucked out of the battery itself. On the plus side, if I got a 2nd battery, I don’t need another inverter, as you can just daisy-chain em. The red switch is an emergency cut-off.

Back upstairs in fusebox land, you end up with an extra widget called an EM115 that takes up one fuse slot. Its a fancy management thing that seems to be required to check everything works:

Connected to this widget is something called a CT clamp. This is a thing that wraps around a cable and tries to detect the power load going through it, and the direction of it. I already have one, used by my car charger in the garage, to ensure the charger never overloads the house grid connection. This new one is connected to the EM115 and the battery so the battery can tell whats happening with my grid connection:

This information is important because the battery uses it to do cool stuff. For example, you can say to the battery ‘Try to never export power to the grid!’, and then on a day where the solar power is high, and your consumption low, the CT clamp tells the battery that we seem to have a negative power flow (exporting to the grid) of X watts, and it therefore tells the battery to soak up X watts of power to balance it out and ensure any ‘surplus’ solar is kept in my house and not given for free to the evil energy supplier!

Whether or not this is what you want your battery to do is dependent on your circumstances. We got our solar panels 12 years ago on an early ‘feed-in-tariff‘. This means that we get credited a nice amount of money for each kwh of power our solar generates, regardless what happens to it, and we get a ‘deemed export’ of 50% of that. This means if our panels generate 10kwh, we get paid 10xFIT plus an extra 5xExportTariff. The actual export tariff is super low, so it wouldn’t be a good deal anyway, but as our tariff doesn’t measure the actual real export, we are kinda gaming the system a bit there by trying not to do any :D

For people with newer solar panels, they may actually have an export meter, and genuinely be paid per kwh they export. So why would you not do this? Because the very very best rates you get to export are insulting. I’ve seen a high of £0.07/kwh which is an insult considering that companies SELL you the same power for £0.24/kwh… Thus even if you do have an export meter, it makes more sense to keep that power in your home and use it later rather than sell it, then buy it back at a huge loss…

For people with normal jobs this is even more acute. In the UK summer your home solar will generate tons of power while you are out at work, which it sells for £0.07 for you to buy back in the evening to run your home at a huge loss. Sod that!

But enough about the economics! lets look at the real reason anybody gets a home battery – fun charts to look at:

This is one of the many views you get from the givenergy website. It also has app for your phone, obviously. This is my first day with the battery, and I screwed a lot of stuff up which I will explain. As part of its first-run ‘calibration’, the battery gets filled to 100% from the grid. Thus it was at about 80% full around midnight. I had then stupidly told it to fill to 100% between 00.30 and 4.00AM which is when my electricity is 75% off. It did this (see the top purple line going up to 100%) by importing that power from the grid. Thats the red section below the base line on the left, and the purple is it filling the battery

The red is a bit higher than the purple (well…below…) because I also needed *some* power to run the house even overnight.

I had screwed up twice because I’d told the battery to only discharge from 4.00am onwards, so between about 1.30-4am you can see those red downward spikes showing me importing power to deal with our energy demand (green upwards). Then from about 4am onwards, that disappears and we are running entirely from the battery since then (including now). Thus the battery level is slowly running down (top purple line).

The UX of the app is not amazing, hence my screwups, but I think I’m getting the hang of it now. You can see on the RHS of the chart some downwards purple and some upwards yellow. Yellow is the output of our solar panels, and for a period there it exceeded our energy usage, so the excess was dumped back into the battery. Working as intended!

You also get real time data like this:

This sort of thing can drive someone insane, because very often the numbers do not match up exactly, and you start to wonder if everything is broken. It isn’t, but you have to get your head around two concepts:

Firstly, the accuracy is not perfect. We do not have an export meter, so we are relying on some pretty rough measurements of power flows being detected by a loosely attached metal clamp on a cable. Thus you have to allow for a bit of a fuzz factor.

Secondly, the battery response is not instant, but has a slight lag. This is REALLY important, and can cause confusion. For example, say you are consuming 200w of power, and the battery is happily supplying 200w from power you saved up earlier. All is good. Then suddenly you switch a 2,000w kettle on. That power HAS to come immediately, and the battery is not set to do it, so for a few seconds, the extra 2,000w comes from the grid. The clamp detects this, tells the battery, and the battery ramps up to output 2,200w instead. Kettle finishes boiling, but the battery is still at 2,200w. The excess automatically flows to the grid. The clamp then detects that, the battery is told, and ramps down to 200w again.

That all sounds ok, but what it means is we briefly imported 2,000w from the grid, and then later briefly exported 2,000w as well. In the grand scheme of energy flows, its a minor deviation, but that is why there is not a perfectly flat line between 4am and now, with zero grid import or export. The import/export is VERY low compared to normal, but it still does happen.

If you have a smart meter in-home-display you will already be fairly familiar with this. In theory your TV will use maybe 200w for example, but you only need a helicopter to explode in an action movie, with a big loudspeaker-driving bang, followed by a loudspeaker-minimal silence, to see that the per-second power draw of almost everything is very, very variable.

I guess this is early home battery technology. Maybe eventually this sort of thing will be somehow perfect with sub-second response times. Maybe thats really hard on some components and electrically tricky. I don’t know, I’m just some guy with a battery in his cellar.

ONE MORE THING

You *can* wire a battery to be a backup situation if your home loses power due to a cut. We did not do this. Initially I had been told this could be done. I was then told that it can only be done for certain circuits in the house. I was THEN told it means adding yet another fusebox, at which point I bailed. Our kitchen already looks like a nuclear power control room.

One of the undiscussed issues with home battery as backup power is that the actual output RATE from a battery is quite low, in this case 3KW. 3KW is a lot, until you have a router, 3 wireless boosters, a TV, streaming stick, home theater unit and a subwoofer plugged in… and then turn the kettle on. Basically you run out of headroom, and things will start cutting out. Home batteries are great, and can provide a lot of power, but the big old electricity grid can deliver 100 amps at 230v and thats a lot of juice. If you want to be able to power everything, including kettle + electric cooker and also charge a car and do everything else all at once, you need some serious big-ass inverter, and frankly, powercuts here are rare enough that I decided to just not care.

I will do (inevitably) another post in a week or so, when I have long term data, and the energy bills to prove it. I’m still in the exciting early days of trying to find an excuse to go get something out of the freezer (also in cellar) as an excuse to gawp at my battery…

For those curious, the total installed cost was £5600 plus vat @20% £6720.

Optimization for fun!

I am well aware that my game Democracy 4 is not exactly slow with huge framerate issues. However, optimization is fun! or at least it should be, but in practice, getting profiling to work on remote PCs is not exactly easy. I have basically used every profiling software imaginable and still have not got one that I think really does the job well…

I have basically wasted about an hour today trying to work out why I couldn’t get the intel vtune amplifier stuff to work with event based profiling and get rid of this pesky error that was clearly nonsense about ‘not able to recognize processor… until I finally realized that I actually have an AMD chip in my (relatively) new PC so…yeah… That drove me to try out the AMD uProf profiler, which is something I had not used before.

It took me a moment to realize that this software, good though it is, does not suggest to you ‘hey if you run me in administrator mode I will show you 50x more config options’ but luckily I worked that out. My first act was a brief run of Democracy 4, starting a new game then immediately going to the next turn. In the list of functions taking up all the time (and ignoring windows system functions) I get this list:

Which is about what I would expect. The game is implemented as custom-coded neural network structure, hence the terminology. Mostly everything is a neuron, and most of the processing is where each neural effect (the links between neurons) processes its equation, and then neurons do some math on their inputs and outputs.

The inner machinery of the neural network ultimately comes down to that top item there: 

SIM_EquationProcessor::Interpret Value.

This is code that basically takes those equations in the game’s csv files like this:

StateHealthService,0-(0.4*x),2

And actually calculates a value from that. There are 2,000 voters with about 10 connections each, pre-processed on a new game 32 times, so thats 640,000 equations right there, plus all of the actual simulation stuff layered before that. In other words, that equation processer probably runs a million times on a new game, and the equation might have 5 values in it, so max case is 5 million values get interpreted when you click on ‘new game’.

Can I speed it up?

First step is to see how stable these values are any way, so I’ll do an identical profile run and check that the +/- errors on different profiling runs are small…

I think thats pretty close. I definitely have numbers here that are in the same ballpark. So now lets try some optimisations to speed this puppy up. Looking at the top function with a double-click gives me a whole bunch more data:

The function is much longer than this, but thats mostly catering to relatively rare edge cases. Looking at the bits that actually have numbers on it show pretty clearly that its pretty much all about the pesky strcmp() call. A separate piece of code has already parsed the full equation of 0-(0.4*x), so I have a bunch of char buffers for each variable, declared like this:

char Vals[MAX_VARIABLES][32];

The thing is, do I need the overhead of calling strcmp() when I am only really checking for whether the first letter is x? Sadly I cannot JUST check that, because that would prevent we having a named variable starting with an x. Lets imagine this equation:

0-(0.4*xylophone)

Obviously not very likely, but theoretically possible. If the length of the buffer was 1, and the first letter is x, then thats a hit, but the question is, will inlining 2 manual checks be faster than a strcmp function call? Lets replace that code with

if(Vals[valindex][0] == 'x' && Vals[valindex][1] == '\0')

And check out the results:

Hmmm. Actually WORSE as far as I can tell. So it looks like whether we strcmp or not, just checking the value of two bytes at that point is slow. probably because its not immediately available memory? Its notable that the code at line 232 is super fast by comparison, as its just checking a bool value we cached earlier. Maybe I should try that? When I parse the function, just keep a bool for each Value, saying if its ‘x’ or not?

Whoahh. This looks like a pretty major speedup. 326 cycles versus 1,143. What the hell? why didn’t I do this earlier? Lets look at the line by line…

This is awesome. I then tried to make this code inline, but it seemed to not make things any better. I haven’t fully explored uProf yet, but it does do cool flame graphs:

Profiling UIs are great fun :D