The good thing about building an engine just for your game is that it lets your game do things other games can't (and it's often fun - or at least educational).
The bad thing about building an engine just for your game is that if you want to Frobber1 something, you first have to build Frobbering into your engine.
The ugly thing about building an engine just for your game is that building an engine means making a lot of choices (what can even be Frobbered?). And - especially when you're coming from a non-game-development background - it can be hard to tell whether you're making the right choices.
This update is more programming-oriented than usual, so if that isn't your jam then feel free to skip. No gameplay-related news here. Back to our regular update style next week! (hopefully)
The Questions
I was pretty happy with my speed of development until about 6 weeks ago.
6 weeks ago is also when I shifted focus: instead of figuring out how atoms and moving bodies should work, I started implementing a bunch of kinda-boring stuff every game needs - like loading levels.
Initially the slow down was because those things needed under-the-hood game-engine-y bits and pieces (like making asset loading work for the web build).
But then those slowdowns made me wonder whether writing an engine was a sane choice in the first place: after all, standard advice is "don't write an engine"1, and a good chunk of what I've spent time on is "free" in other engines - everything except atoms and their interaction with rigid bodies, really. 2
The Crisis of Faith
I'm writing the game/engine in Rust, and 3 weeks ago LogLog Games published Leaving Rust gamedev after 3 years. 3
I have experienced all of the problems mentioned by the author,4 but I had assumed workarounds existed for them or that they were manageable.
That nagging back of my mind question about writing an engine suddenly became very front-of-mind: did I make the wrong choice in choosing Rust? Am I insane for writing my own engine?
On Choosing Rust
I chose Rust (and a custom engine) because of three requirements:
- Performance: I need fast performance for the atom simulation - which means using Rust, C, C++ or (maybe) HPC#. 5
- Determinism: when the game runs, I want each step of the game's simulation to result in identical output, no matter what computer it's running on. 6
- Learning: implementing game-engine-things from scratch will teach me more about how games work under the hood, so I can make better games. 7
But! I missed the key requirement for making a fun & engaging (vs technically-correct) game:
- Iteration speed: be able to try game ideas quickly to find the good ones and throw out the bad ones.
And as LogLog Games points out (at length), iteration speed in Rust kinda sucks:
- Hot reloading (to make a change & see it in the running game) isn't great. There's a way to reload code but slow compile times make it less useful.
- Quickly hacking an idea together is hard: Rust errs on the side of correctness over convenience, even when the correctness is unnecessary. 8
- Rust game frameworks/engines are immature so you need to implement a lot yourself. Instead of drag-and-dropping some rectangles around on a 2D editor UI to position them, I had to think about and integrate full-on level loading.
The Search for Answers
Can I meet all 4 requirements? Well, the "learning game (engine) dev" requirement conflicts with iterating quickly, so: no.
But I can give up some "learning" for some "iteration speed".
And so (here I condense 3 weeks of research & investigation into a few sentences) a reasonable balance of the 4 requirements is to still write the core game simulation in Rust (for performance and determinism), but use an existing game engine/framework to power the undifferentiated game-engine pieces (gaining back some iteration speed): things like audio, rendering, input handling, windowing, etc.
The 3 major game engine options are (in order of "most mature" to "easiest to integrate Rust") Unity, Godot, and Bevy - so I'm leaning towards Godot as a reasonable balance.
I haven't actually done that integration yet9, but I feel a lot better about having that plan in place.
Better Iteration in Rust
Some of my investigation into bending Rust towards faster iteration times also helped:
- I shuffled some code around10 to bring re-compile times down to under 2 seconds. It takes about 2 seconds for me to decide to run the game and switch to it, so now it feels like I can see the effects of a change instantly (rather than waiting 10ish seconds).
- I figured out how to stop my editor's automatic re-compiles from interfering with the automatic recompiles I use for running the game. 11
- I realized that my code editor's Rust plugin has a bunch of extra functionality that you're supposed to map your own shortcuts for (like "smart enter" and "move item up").
And, I reorganized the game logic to use an Entity Component System (called hecs12). For the coders: basically instead of having big objects with functions attached, you have a list of numbers (entities) and each number has one or more chunks of data ("components") that can be accessed; you can't get at the data except by writing functions ("systems") which read data from the world using using those entity integers or asking the world for the records for all entities having (or not having) certain records.
Anyway, Entity Component Systems are often pushed for performance benefits, but I mainly adopted one because it sticks a lot of your game's state into one place, and that's more convenient when you want to hack something together. (I was already composing objects together to get behaviors, but that composition did get a bit nicer too.)
One thing I haven't tried yet is getting actual code hot-reloading working - maybe next week.
Gameplay-identical web build
This week's playable build should be exactly the same as last week's - at least as far as you can tell.
Well, maybe in the course of my invisible work I snuck in a bug or two - do let me know if you find any!
Frobber is not a real word, so you can pretend it means whatever you want.
Well, rendering pixel art without distortion at arbitrary scales is also not out of the box in engines I've looked at. But I could port that over to any half-decent engine.. probably.
Though really, the article says Write Games, Not Engines (i.e. write what you need to in order to make your current game work, don't make an engine for all future games you might want to make), which I am actually doing. Still, it's easy to see that if you make a game in an existing engine, the game's going to get finished much faster.
If you're at all interested in using [Rust] for developing games, it's worth reading. But the gist of it is that Rust's correctness oriented design gets in the way of iterating quickly to "find the fun" in a game idea.
Even the very specific physics bug caused by a bug in Rapier's (the rigid body physics library) implementation of creating a polygon collider from a (~)bitmap, funnily enough.
HPC# is High Performance C#: a subset of C# created by Unity, which their Burst compiler accepts and is able to compile to native code. Technically they can compile a lot of C# to native code using IL2CPP, but that still supports a bunch of features that make the generated code slower (e.g. Reflection); HPC# puts stricter limits on what you can do (e.g. no reflection), and in return can generate code that's pretty close to C/C++/Rust speeds.
This "cross-platform determinism" is necessary to have any shot at making my multiplayer fever dreams work, but regular determinism also helps with debugging. It's harder than it sounds because (perhaps surprisingly) floating point calculations are not guaranteed to give the same results across different CPUs or operating systems. Unity's HPC# does not give this guarantee, for example, and most physics engines (or as far as I could tell, all the popular ones - except Rapier, which I'm using) don't either.
I believe that learning "the layer below" the layer you're working on tends to result in better outcomes. For example, if you're making a website, understand how the website gets shuttled over the internet to your viewers.
in single-threaded code or code with a very set structure (like game loops for simple games), you can often know that certain things aren't going to happen. In those cases, Rust will still force you to write your code defensively to satisfy the compiler, just in case you refactor your code to do something dangerous later. That's great for maintainability, but for cases where I'm throwing away the code I'm writing, I won't care about maintainability. When I started this project I hadn't written much "unsafe" Rust, so I thought that the unsafe let me opt out of at least some of those checks - but it doesn't actually, unless you want to risk Undefined Behavior. (I wish there was a language that explicitly designed for a "hack, harden, maintain" workflow..)
It's not laziness! Godot's Rust integration's web builds support is still a work in progress. I don't want to stop publishing web builds just yet!
Serde (the saving/loading stuff to/from disk library I'm using) generates a ton of code (really, LLVM Intermediate Representation for LLVM to compile) for reasonably large serializable structs like that generated by LDTK. Moving that into a separate "crate" (compilation unit in Rust terms) meant that it didn't need to be recompiled by LLVM each time. Also, turning link time optimization off entirely for my development builds made a big difference too.
For Visual Studio Code, Rust Analyzer by default uses the same target/ directory for compilation output as everything else; if you set "rust-analyzer.cargo.targetDir": true, in your VSCode settings, then it will instead use a sub directory. Only one compilation process can use a target directory at once, so using a different one makes a real difference if you have bacon or cargo-watch running cargo build for you in the background on each change too!
Rust gamedev folks might be asking: why hecs rather than bevy_ecs? Partly because hecs enforces less structure on your game (making it easier to remove if it doesn't work out in a few weeks), but mainly because bevy_ecs is wildly undeterministic: 1) unless you fully specify the ordering of your "systems" (functions that implement your game), the ordering of execution of your systems is subject to change up to 60 times a second. 2) the order of iterating through entities is unspecified, because there's no way to clone a bevy_ecs World (collection of entities & their data) in such a way that it preserves internal state. You can work around each of those things but I want my life to be easier, not harder.