Game Design, Programming and running a one-man games business…

Still balancing Democracy 4 with data

Posted on January 9, 2022

My current game: Democracy 4, currently has very good steam current reviews and all reviews scores. As I check right now it sits at 90% positive recent, 85% positive all reviews. However, 90% is not 95%, and any improvements to the game are bound to translate into higher scores, longer play times and better sales, and as balancing the game requires a huge amount of data analysis and thought, and is therefore what I consider fun…lets do some more!

Because it is a ludicrously complicated game, the only possible way to ‘balance’; the game is statistical analysis. There is no way I could hire a team of QA people to give me feedback on where the game is unbalanced, because the number of play styles and strategies, and political opinions, and then the different countries, difficulty settings and other options would mean even if I had 100 people analyzing 20 games a day, its still ludicrously trivial compared to the actual ‘problem space’ that the game represents.

Thankfully, the game sells well, and if I pick yesterday as an example, we clocked up a total of 3,754 games played by players in one day. These are proper paying customers whose stats don’t lie, so analyzing the data from these playthroughs is invaluable. I do not collect ALL possible data (that would strain even my dedicated server, whose current database size is 14 GB), but I do collect some interesting snippets (anonymously).

I rely on my own intuition and feedback in reviews (good and bad) combined with forum posts to give me a feel for areas where the game needs improvement. A topic that seems to come up a lot (with justification) is that the first term of a game is pretty well balanced, but if you ‘survive’ to a second term, things get easier and easier and the game loses its challenge. To check if this was really true, and look at some possible causes, I collected stats by each term and got this:

T1 Wins = 54.19 %    T2 Wins = 79.23 %    T3 Wins = 83.97 %   

Thats term 1,2 and 3+ win rate for game version 1.45. Its pretty clear that the intuition is correct. The game gets easier the further you go. The big question though is exactly WHY does this happen. In order to collect relevant stats I needed to think about what factors could make governing a country too easy. Maybe GDP gets high and stays there? Crime gets solved for good? Technology gets ahead and stays there? Unemployment solved for good? I collected player stats at the time of each election and plotted them over 1,2 and 3+ terms to get these stats:

and also…

So it seems that the stats do not lie. Multiple things seem to get good and… stay good. Education seems a bit easy to raise after 2nd term, but maybe thats because a booming economy allows for infinite spending on state provision? Sadly I do not have data for that, but its clear that GDP has an inexorable rise. Its also interesting to see that crime steadily declines, especially violent crime.

Stats like these are invaluable. Democracy 4 has a bunch of self-balancing mechanisms that are not artificial game constructs but based on what appear to be real-world auto-balance phenomena. It does seem to be the case that skyrocketing GDP in the real world can drive massive inequality (relative poverty…. even if actual absolute poverty declines). This rise in inequality can give rise to higher rates of crime, and also to political dissatisfaction.

Higher GDP and Technology can be a sign of increased industrial automation, which should really cause big spikes in unemployment, and thus also ineuqlait6y, but these do not seem apparent. Luckily, I have control of every equation in the game, so after reviewing all these stats I could make a bunch of adjustments and hope to see a budge in the direction of a more balanced (ie: more challenging) late game. Lets look at the new stats for build 1.46:

T1 Wins = 54.54 %    T2 Wins = 76.46 %    T3 Wins = 83.06 %   

Ok… so slightly better T2 (a bit harder…. 76% win vs 79%) and very marginal improvement from term3 and beyond. How did my actual core stats change:

If anything WORSE for GDP, although I did make unemployment worse, and a slight term 2 boost to crime, but fairly trivial. second row:

Ok… so equality IS now harder to raise in later terms, which is a slight improvement, but basically I have been way too meek in my adjustments and need to ramp those changes up. I also started tracking some additional stats:

This is new data for build 1.46 without prior data to compare to, but I felt it important to start measuring. Winning is the value of a special hidden variable I use in the game to adjust difficulty. That percentage DOES go up over time, which makes sense, although its clearly not hitting the upper 90s, so I could simply take into account consecutive election wins as some input into calculating its value?

Turnout is more of a curiosity, but vote percentage is much more interesting. In the *ideal* game of Democracy 4, the player scrapes narrowly through on each election, excited to view the results, and relieved to know they won another turn. 52 and 58% feel about right, but averaging 74$% is pretty worrying. Perhaps cynicism is not a strong enough effect here, especially given that it clearly drops between terms 2 and 3! (in the real world show me a president who people did actually become less cynical about over time…).

So the main thing is that I have useful stats and a system to compare them and check I am heading in the right direction. The game clearly got a bit more balanced between games 1.45 and 1.46, so here is hoping that being able to spend more time post-release on balancing versus features will allow me to improve that even more…

Speeding up Democracy 4 simulation processing (atof is slow)

Posted on December 29, 2021

I just made a major speedup to the loading/next turn/new game code in Democracy 4, and thought I may as well share the gains with readers of this blog :D. It kind of makes me look a bit like an idiot to admit this is a big speedup (I should have known), but anyway, knowledge sharing (especially on optimization) is always good.

Fundamentally, the code in Democracy 4 is structured like a neural network. Without going into tons of details, every object in the game (policy, dilemma option, voter, voter group, situation…) is modeled as a neuron, which is basically just a named object connected by a ton of inputs and outputs to other neurons. You can run through the inputs and outputs, process the values and get a current value for a neuron at any time, which is done for every one of them, every turn.

Also… when we start a new game, I need ‘historical’ data for each value, so the game pre-processes the whole simulation about 30 times before you start to give us meaningful background data, and to ensure the current simulation sits as a reasonable equilibrium.

Those connections to neurons should probably be called dendrites or whatever, but I call them SIM_NeuralEffect. They contain basically the names of a host and a target (resolved to actual C++ pointers to objects), and an equation explaining the connection, and some other housekeeping stuff.

At the heart of it all, is an equation processor which lets you write this:

OilPrice,0+(0.22*x)*GDP

And actually turn it into a value for that effect, given the current situation. The Equation processor runs each turn, on every neural effect, and there are LOTS of them. Thus, if the equation processor is slow, its all slow.

I just installed a new version of the free vtune profiler from intel. Its not recognizing my ultra-amazing new chip, so only doing usermode sampling, but nonetheless it draws pretty flame charts like this:

Before I optimised

This is showing the code inside that 30-turn pre-game processing called PreCalcCoreSimulation. Lots of stuff goes on, but what I immediately noticed was all this atof stuff. Omgz. Thats a low level c runtime function, not one of mine, and it seems to be slowing down everything. This is a HUGE chunk of the whole equation processing code. How is this possible?

Now, you may think ‘dude, atof is pretty standard. No way are you are going to be able to make that code faster’, to which I reply ‘dude, obviously not. But the fastest code is code that never runs.’.

All those calls to atof are absolute nonsense.

Looking back at the equation above (OilPrice,0+(0.22*x)*GDP) there is obviously some stuff in there which is volatile. I do not know what the current value of x or GDP is, so I will need to grab their pointers and query them when I process the equation, but the rest of that stuff is static. That * is going to remain * and that 0 and 0.22 will remain fixed too. This is the key to a roughly 33% speedup of the whole processing in the game.

I actually did know to look into this, and I do not do manual text parsing of the equation each time. I parse them equation on startup, and stick the various values into buckets, so I am not wasting time each turn. But one thing I had not done is store the atof() outcomes. I was still storing variables[0] as ‘0’ instead of just 0.

Now you may think atof is fast. Its fast enough for most cases, but its WAY slower than just accessing the value of a floating point number thats already in RAM, and cached happily in the equation processor itself. Here is the new diagram:

Faster!

The difference is, (on my superfast PC), for the whole precalc simulation function: 0.85 vs 1.50 seconds. This probably makes me sound pedantic as hell, but I’m rocking some stupidly new and pricey PC, so there are likely people playing D4 on laptops a fifth the speed. I might be knocking a whole 3 seconds off the new game time for some players!

Also, and worth remembering, I just saved doing a ton of processing, which means a ton of CPU time, power and heat. If you can make your game run more quietly, more coolly, and faster on players PCs, you absolutely should do it.

Re-framing social media

Posted on December 26, 2021

I haven’t been on twitter for a few weeks, and its been good. Here is why.

I started doing a lot of thinking about the internet, and social media specifically, and I really do not like the current state of things. I read a truly excellent long blogpost about the bloat in web pages these days, that would be funny if it was not so tragic. It reminded me of a time when people in IT gave a flying fuck about performance, efficiency, and common sense, rather than ‘user engagement’ ‘metrics’ and ‘stock options’.

The moment you step aside from social media, you start really seeing it properly for the absolute dystopian hell that it is. Anybody with common sense would absolutely leave it immediately for their own health, and no doubt many do, but the rest of us who are still there do not even know that happens because… we only know about what happens on social media.

With thousands of ‘followers’ on social media, we are part of a rich ecosystem of always-on, always updating action. The fact that 10 people we knew just stopped posting entirely and disappeared will not even register to us because…oh look a kitten video.

Lets be perfectly clear. Twitter is a private company, not a service. Its sole reason to exist is to make as much money as possible. if that means destabilizing governments, wrecking democracies, driving people to suicide, and destroying the mental health of an entire generation or two, then nobody at twitter is going to care one bit. They have shown this to be true time and time again. Its still rife with abuse, hatred, misinformation and crypto-scam troll-farm bullshit.

If you are anything even vaguely like me, then you will know, deep down, that twitter, and similar sites like Facebook, are absolutely disastrous for your mental health. By participating in these sites you are putting up a big ‘open for business’ sign next to the emotional center of your brain, with an open invite to people with extreme, warped views and aggressively evangelical personalities to come unload their hatred and prejudices into your skull 24/7. Its the most destructive drug imaginable. Even with heroin or cocaine, you actually run out at some point, and need to call your dealer. With social medial, the dealer is right there, right next to you, with a constant fine-tuned supply being injected straight into your veins all the time, for free. Best of all, hardly any of you, statistically, will kill themselves, so the dealer can keep it up for longer.

The whole idea of craving more ‘followers’ on twitter is absolutely insane. Jesus had followers. I guess Gandhi did too. Maybe Martin Luther King. Certainly some horrible dictators also had many followers. But does someone who makes videos about knitting on the internet need ‘followers’? and do they need to know the exact number of them, all the time, updating hourly? This is objectively insane. Its a skinner box for faked charisma. Its pure evil. We cannot cope living like this.

I think I have 10-12,000 twitter followers the last time I looked. I don’t think its changed for years. Many are likely bots. Its depressing that I even know the number to that level of accuracy. We should not want to know. We should not want to play such a game. Many people I know DO play this game, because they are content creators (indie game developers) and they think this will raise their profile and sell more games. I think this is a mistake.

Here is a top tip: Take all the time, effort and energy you spend tweeting, replying, re-tweeting and scrolling on twitter and spend all that effort instead working in a coffee shop waiting tables. Then take the money from that 2nd job and spend it on advertising or improving your content (whether its indie games, art, books, whatever). Your ROI will be WAY better.

We all know people who ‘got famous through twitter’. Of course we do, because twitters algorithm will take any story along those lines and promote the fuck out of it. How many people do you actually really know personally who owe their fantastic success to their twitter account? Do you actually ever do any analysis on whether or not being viral on twitter sells more video games? Let me help you with that.

Here is one of the most famous people on the planet replying to a tweet at him by me:

WHOAH. I bet a TON of people checked out my twitter profile and then went and bought my games on apr 14th 2021 right? Lets see the spike:

Oh dear…nothing… ok Here is a tweet I did on a random whim after I saw a link about it. I didn’t even create the original content.

Ok… now we are talking. Thats pretty cool right. A tweet with 48,000 likes. Not bad cliffski, not bad at all, this is where we see the sales spike:

yeah…I’m kinda not seeing it. But maybe I’m a freak and my steam sales are never spiky. Lets looks at the current Christmas steam sale to check that theory:

Hmmm.

And YES, I get it… MAYBE some small number of people who see you tweeting something viral are going to check your profile and MAYBE some of those people will then be 1% more likely to buy your game in the next year and YES maybe its hard to track but…

I’ve been on twitter forever, and I basically have had about half a dozen ‘viral’ moments, and none of them have led to any noticeable boost to my business. If I could go back in time, never join twitter, and never join reddit or facebook, and get back ALL that time and convert it into more direct work on marketing or developing my game would I do it? Of course.

For those who like the numbers. Lets say those 48k likes represented clicks on an ad. Thats 48,000 clicks at maybe $0.05 CPC because its poorly targeted. $2,400. Assuming I’m due another viral tweet every 10 years, thats $240 a year for my social media addiction. yay?

Apart from anything, I really resent putting any effort into these social media platforms in the past only to realize way later that I’m basically an unpaid content creator for a billion dollar US-owned social media company. At least twitter have not YET started charging people money to ensure all your followers see your tweet (thats coming though…), whereas facebook blatantly flipped that switch years ago.

Twitter is disastrous for your mental health, and sites like facebook and reddit are even worse. Actually RANKING peoples opinions with up and downvotes? Are we serious here? Imagine going round a friends house for a few beers and a chat, and every time anybody said something people all held up up and downvote paddles. Does that sound like fucked-up psychotic behavior to you?

Full disclaimer: I advertise on social media. It works reasonably well, I wont stop doing that. But I left facebook a few years ago with no regrets, and only maintain facebook and reddit accounts because my games have pages on both networks. I’m not on tiktok or snapchat or instagram.

I’m calmer, and happier since I stopped using twitter, and I have a very, very hard time seeing any way in which its affecting my business. Frankly I have more time now and less distractions to actually improve my game. If you follow me on twitter you may see the odd post for some big event that I want to ensure any press following me see (like the upcoming release of Democracy 4 from Early Access), but in terms of casual day-to-day tweeting and banter on twitter? I think I’m going to save that for real-life friends from now on.

What exactly is it that you do here?

Posted on December 13, 2021

I’m in the middle of making deals with two different sets of what we shall call ‘contractors’ here, although in both cases they would probably be offended at the term, because somehow people get very exercised at pretending they are not working for somebody when they are in fact, working for somebody… (you can tell they are, because that person pays them)

This process is absolutely infuriating to me, because what my ‘probably on the spectrum’ brain thinks should happen, and what actually happens is so completely and utterly different. Here is how I expect things to go when I need some work doing by a third party.

  • Cliff searches internet for lists of people who do X.
  • Cliff sees the prices each contractor quotes to do X
  • Cliff selects the best price/experience/talent combination for his needs and a contract is signed.

Thats not vaguely what happens, because what actually happens is more like this.

  • Cliff searches internet for lists of people who do X.
  • Cliff finds that each individual has a contact form, but no actual prices.
  • Cliff fills out contact forms and sends emails to people who randomly guesses are affordable, but has no real clue
  • Strange middleman Y who works for X, replies and suggests that cliff ‘hops on a call to discuss the job with Y, who I should repeat is not X, and is apparently somehow ‘front-running’ the whole deal making process.
  • After declining this social invitation, Y is now prone to asking cliff was his budget is, or what his expectations are, or what will happen to the work after its done and other such bullshit.
  • Cliff gives up and just doesn’t get the work done at all, in despair.

The whole thing just reminds me of office space:

A classic film, with a great scene where someone whose entire job description is that they serve as a pointless middleman between the customers and the workers, desperately tries to justify his existence and continued employment.

Fuck this.

If I go to a shop to buy some bread, it has a price on the product. I can buy it, or not buy it. Thats not to demean the bread, it could be a cheap sliced loaf, or an exotic sounding freshly baked warm artisanal loaf that costs 5x as much. Thats fine. I get that quality costs money.

What I don’t get, thank fuck, is some salesperson leaping in front of the bread and saying “Hi there, you seem to be interested in working with our range of yeast-based consumable experiences, what kind of budget do you have in mind? I can talk to our bakers and see if they can provide something that meets your expectations, but first let me talk a bit about what a great deal you will be getting if you choose to go for a sourdough option”.

I do not care if you sell bread, artwork, music, PR services, animation, full motion video, advertising, I really do not give a fuck. All I want is a price, and as few middle-men as possible.

I recently bought a pretty pricey car. I did it with one click on a website. I strongly suspect the only human involved in me getting delivery of the car will be the person who physically hands me a key to it when its in the UK. I did not need, nor did I pay for, a middle-man to explain to me what wheel options exist or comparing the different trims. This is all on the website.

Imagine that, a website where you describe the product or service is plain text, saving us all the futility of inserting some useless, waste-of-space middleman into the transaction.

BTW, some services (game dev related) I recently tried to get would cost $5-10,000. The middlemen responsible for ‘representing’ those service providers did not even reply to me. 10 seconds google will tell you that yup, cliffski can probably afford those services, and isn’t likely to fuck around, or fail to pay.

In this case, not only did the middle-men add no value, they actually destroyed it, and the gullible fools who hired them will never know about all the money they should have earned easily, but missed.

And here is the thing: some people think hiring an agent will make them MORE money. No. In my case, not only and I likely to pay WAY LESS for someone with an agent (because I know the real cost of their services is way lower, due to the agent needing to be paid), but its actually a net negative, because by wasting my time talking to a pointless intermediary, you have actually provided a WORSE service for me.

Plus I KNOW if I have to haggle with someone over pricing, that its because they are trying to size me up and squeeze more money out of me. Its the online equivalent of someone quoting more to do your plumbing because they notice your nice car. Its insulting, its rude, and aggravating, and it just means you are training your customers to be aggressive, confrontational, penny-pinching bastards. Is that how people want to do business?

Needless to say, I don’t have an agent. If someone wants to contact me, cliff AT positech dot co dot uk is my email address, Direct to me.

Lessons from 41 years of programming

Posted on November 26, 2021

I don’t blog much about the nuts and bolts of programming, because I know enough to know that others know more than me, but increasingly I’m aware of how other people code, and feel that I probably do have *some* worthwhile advice on the topic. Not many people have been programming for 41 years, especially given how coding seems to skew young, so here we go…

First some background, I started coding in sinclair BASIC on the ZX81 aged 11. I took some time away from code to try and be a musician, and then worked in IT support, but I have worked as a full time programmer for the last 25 or so years. After a while its hard to keep track! I basically code video games in C++. Thats it. I know some php, but not much. I can probably remember some BASIC, but not much. Yes, I’ve been coding 25+ years in a single language.

Lesson #1 Learn a single thing well.

I sometimes do some stuff that would probably be a bit easier in C#, or Rust, or Java, or probably some other language I have not even heard of. I don’t care. I am a massive believer in knowing how to do one thing super well. This is very unusual, because these days, coders are hired as a shopping list of buzzwords. Adverts for coding jobs amuse me no end. They always want someone with experience of 10 different languages. Did the lead programmer have a problem deciding which to use? When you want a coder who knows 10 languages, you get someone who is mediocre in lots of different ways. Thats not ideal.

The ‘familiar with 10 languages’ trope is common because non programmers understand it. A coder with 10 languages must be more skilled than one with just 1 right? To a non coder, its all unintelligible gibberish anyway, they have no way to tell a really GREAT c++ coder from someone who just knows a bit of syntax. Recruitment consultants and idiotic managers without coding experience have created a world where everyone is truly rubbish at 100+ different programming languages.

We do not expect this in other fields.”Doctor wanted. Must be a throat expert / hearing expert / heart specialist / Neurosurgeon”. We also manage to get by in our lives speaking only a single language. Sure, french is more romantic, German more precise, Spanish more passionate, English more subtle, but we don’t change languages constantly in our day to day lives! Tip: If you work for yourself, learn ONE language super well. Only learn others if they are absolutely essential for the task in hand. You can do almost anything in C++.

Also note that being really good at one language does not mean learning every possible feature (see next lesson), but it means using that language so MUCH, in so many different circumstances, that you know the best way to get things done using that language. (by best I mean reliably, efficiently and legibly).

Lesson #2 Language features are optional.

C++ lets you override operators. It also supports templates and the stringizing thingy (## i think?). I don’t use any of them. I use maybe a quarter of C++. Most of the newer stuff seems like solutions looking for a problem. Its FINE to just stick with the feature set you find to be most usable for you. Nobody is going to point at you at parties and mock you in front of the opposite sex because you don’t use the whole range of C++ features in every program.

There are multiple ways to handle files in C++. fopen, iostream, CreateFile() etc. I use the old fashioned fopen() stuff. I know the syntax super well. I can type it as fast as I can speak. This is fine. I’ve never had someone leave me at the altar for not using iostream.

I am constantly told by people I am reducing my productivity by sticking with old, outdated and cumbersome systems when new-fangled ones have apparently done the same but better. Without exception I am more productive than everyone who tells me this.

Lesson #3 Legibility is the goal.

Coders start off as n00bs with no confidence or idea how to do things. They then go through a ‘gunslinger’ phase, filled with the coders equivalent of testosterone, where they write ‘impressive’ code, that uses cool obscure language features or fancy techniques to do whizz-bang stuff that impresses other coders at parties. This phase lasts about a decade, or maybe longer if you are in a big company with bosses that need impressing.

The ultimate stage for a coder is to realize that ‘impressive code’ phase is just so much adolescent bullshit. The goal of code is to be reliable, efficient and legible. That last one really matters. If there is a way to perform a task that is simple, clear, and a way that I have used 100x in the past, then thats the way I do it. There may be a ‘cool’ way to do that task that involves callbacks and threading and whatever the heck microservices are… but why over-complicate things when you can just write simple code that you will understand instantly on reading it in 10 years time. 99.99% of the impact of your code is on the users, not the other coders who might glance at it. My code probably looks simplistic, very literal, very accessible for even non coders to read. I’m happy about that.

Lesson #4 Consistent Formatting

No 2 coders ever agree on how to format code. I have my own opinions which I shall not bore you with, but here are two thoughts: Firstly, be aware that a lot of ‘standard’ ways to format code were dreamed up before we had intellisense and visual assist. If you are still relying on a naming convention that shows you what is a function, what is an enum and what is a variable… then its kind of pointless. We have syntax-coloring now. You don’t need to do that, and can get back some much needed legibility. No more LpSZ_mem_Name.

Secondly…and this is even more important, the goal of code formatting is to minimize the cognitive load on the person reading it. Most programs do complex things, and to debug or add features, you need to maintain a HUGE network of complex stuff in your head as you work. If you are having to mentally translate a lot of different variable-naming conventions and code layouts in your head as you read then you will be amazed at how much

HARDER

to

_read

This

cAn

then

BE.

Our brains seem very good and building up mental translation systems. if you have text in ALL CAPITALS then we adjust to it ok. Pretty much *any* coding standard is fine… as long as you have only one. This is another reason why I hate middleware, because obviously its all formatted differently. Having to chop and change mentally between different ways of doing the same thing adds huge mental overload.

If I ran a studio, the question ‘will you agree to 100% without question always follow the code style guide of the company or face immediate dismissal’ would be the first interview question for new hires.

Lesson #5 Be aware of the overhead.

Almost all modern code is absolute trash, total garbage, an embarrassingly badly constructed towering pile of crud that should bring deep shame on everybody who wrote it. People with a lot of coding experience tend to agree on this topic. I think its inevitable, given that the demand for coders vastly outstripped that number of people with experience. Maybe over time it will be redressed, but I worry that these days coders are just used to code being crap and do not understand its even a problem. You don’t need 20 years experience to post a useless reply on stackoverflow. Sadly.

Its REALLY instructive for you to step through every single line of code in the execution of the stuff you do. If you are an app developer relying on middleware that is closed source… i pity you. A simple step through one instruction of yours probably means running 5,000 lines of bullshit layers of code that you will never see. This is why we all have incredible supercomputers, and yet we stare at progress bars…

If you are lucky enough to have full source access, or have written it all yourself (yay!), then do the ‘step-through-everything’ test regularly. It will AMAZE you how much code seems to be run without actually being needed. The inefficiency of most code is mind-boggling, and its because hardware has outrun software so much that we can write code that runs at 1% efficiency and hardly anyone complains.

Learn how to use a profiler, and go through your code looking at where all the time is spent. Its actually REALLY interesting, and kinda fun. I enjoy spending time in stuff like aqtime, or vtune. If you have not profiled your code, then you have not finished it. You are not a software engineer if you have not done the post-coding analysis of what the code actually *does*.

Lesson #6: Avoid massive functions, and massive source files.

There are technical reasons for this, in terms of compile times etc, but it really comes down to the overall structure of code. If you have a function that is 100 lines long or more… then you have probably fucked up. If a single source file for a class is 1,500 lines long…then you have probably fucked up. (yes, I know about function call overhead. Are you working on spacecraft control systems that have to run at 5,000 fps with minimal power? I am guessing no, so don’t worry about that trivial overhead here)

The most common mistake that leads to buggy, hard-to-work-with C++ code is that classes are too big, and functions too long. A function does one thing. The clue is in the name. What tends to happen is people start off with a perfectly reasonable class and functions, then they add functionality (ha!) and the class sprawls, and the functions sprawl. What was once a perfectly reasonable Update(), becomes 100 lines long. What was once a reasonable Entity class suddenly becomes a behemoth.

This is normal. You now need to work out the cleanest, most sensible way to break that class into multiple objects or derived classes, and change the layout so that the code is done in more single-focused functions. This is not some bothersome task that wastes time… this is literally software engineering. This is the difference between a coder and a software engineer. This vastly increases the legibility and debugability of your code.

Getting code to work is just the easy bit. The hard bit is knowing exactly how to layout the relationships between objects and functions so that everything seems sensible, and organised. Knowing how to do this takes a lot of experience, and is normally the outcome of many bleary-eyed 3AM debugging nightmares where you stare at the big giga-function when you are nested 16 deep into curly brackets wondering what the fuck its supposed to do.

Call stacks are your friend! Call stacks are amazing. I have the call stack always open at the bottom of my screen. I offload my mental model of ‘how we got to be here’ into the callstack window. I find this incredibly helpful. Making sense of the flow of code through the callstack window is 100x easier than keeping mental callstacks in mind as you navigate giant functions.

Conclusion

These are my broad thoughts on stuff I wish I knew earlier. I’m sure lots of people disagree, and thats fine, I’m not starting a political movement here, just sharing my experience. Sadly, the awful state of modern coding is largely outside coder’s control, due to incompetent managers, feature-centric marketing and tech-ignorant recruitment practices. I hope its of some interest to solo coders or lead coders with complete control of how a codebase is made.