The Themes

I am not a professional gamedev by any means, so any gamedev I do is strictly hobbyist pissing around for fun. This means I can afford myself some more leeway with what choices I make or what targets I aim for.

I am a big strategy gamer, or at least used to be. I grew up playing WarCraft 3 and Command & Conquer: Generals, was absolutely bedazzled when Frozen Throne came out, and spent years chasing that high. Somewhere around 2011 I fell into the Paradox rabbithole, and currently have 10,833.6 hours of Crusader Kings II logged on Steam, and several thousand more outside of it through hacked or old versions. [1]

Nowadays, I mostly spin my wheels playing Dwarf Fortress, kind of wishing there was a game that would cater to my specific narrow interests.

Over the past couple years, I've been toying with the idea of making a strategy game of some kind that would do what I wanted it to do. CK2 was good, it hit me like crack, but it still missed the mark somewhat. I'm really more of a sci-fi guy, and wanted something in that direction. Space-based or sci-fi-themed 4x and grand strategy games never did exactly what I wanted them to do: Stellaris was too arcadey and casual about some things, and didn't let you have fine-grained control over systems or, let alone, planets, and went further in that direction the more time passed (I still miss the old Stellaris pop systems where each pop would have a physical location on the planet map); there was Galactic Civilizations that came close, but it was still too far away; Civ-likes that were set on single planets just weren't it, either. The more-than-excellent Star Dynasties was the closest I got out of the new stuff out there, with its combination of a space-based strategy and a dynastic, character-based system basically directly aping Crusader Kings (nothing wrong with this!), but I didn't like the scale of it: you still operated in terms of whole systems at once on a scale that wasn't fine-grained enough.

Unfortunately, the actual number of things that really got what I wanted was roughly next to zero, from what I could determine. Two games that did scratch the unexplainable itch (or rather one game and one mod) were the WC3 custom map Solar Conquest, where you could individually control units on several different planets and fought for control over each individual planet with the ability to have partial control over things, and the now ancient game Emperor of the Fading Suns which involves dynastic conflict over individual planets under a vacant interstellar throne.

Given that I now guessed that there would be nothing to go into that niche (and I doubt that there's a high enough demand to make someone fill it for me), I figured I might want to do some development myself, knowing full well I'm not likely to release anything any time soon, or ever.

My 'ideal' game, in this case, would be a space-based grand strategy game with a dynastic character focus, primarily or even exclusively dealing with humans and human factors, with things like sub-system and sub-planet granularity of control and a feudal system of interdependencies between characters themselves, and between characters and titles. Space feudalism, really. How has this never been done!?

The game will be 2D, since I just like how maps work in 2D (I even turned off the 3D terrain mode in EU5, I really don't care for the look of it). I will probably make it turn-based, for one reason or another (mostly because turn-based games are more intuitive to me and will be easier to develop) [P]

It's not really that well-developed in sci-fi literature either, for one reason or another. Sure, there's Dune, but even then you get less focus on the diplomacy inherent in feudalism. It's very rare to really stumble across a piece of media doing what's basically A Song of Ice and Fire in space! But I want to try and tackle the problem, even if it doesn't result in me changing that.

The Target

Given that I'm not really limited by commercialisation of my ideas, I could play around with arbitrary tech and target whatever I wanted. Given that I picked up a total shitbox (in today's terms) from a local university getting rid of its tech, I figured why not target that?

The target for my game was going to be, at least for the time being, a machine with the following specs:

We really are cooking with some exotic computation gear, aren't we!

Beyond that, I also, for some reason, decided I wanted to eventually distribute this game on floppy disks, optimally just one at a time, so that I could distribute it to my friends and go haha guys look how cool this is! You can play my stupid retro game off a stupid floppy disk! This means that I'll have to make sure that my entire game with all its assets fits into somewhat under 1440KB. [2]

Given that the system is running Win98, it is also realistically possible to target MS-DOS on it, given that Win98 is a DOS-based system. This simplifies the programming part a lot, surprisingly, as there are a lot of high-quality libraries and tools targeting DOS that simply don't have a match on the Win95/98 side of things. This also means that I will not have to do much to test my game, as now I can run it just fine locally on DOSBox and expect it to run on hardware as well (and the inverse direction doesn't need to apply).

The GPU this system has isn't that strong, but we're not doing 3D graphics anyway, and just need it to push pixels to the screen. I'm rendering a 1024x768 screen (how huge!) at 16 bits per pixel, which the GeForce2 MX takes like a champ and doesn't stutter a lot. I also don't want to make a 3D game, and not even just because we're on MS-DOS: I think a proper strategy game or wargame can work very well in a 2D scheme, and gains nothing much from going 3D. We'll be doing tiles from a tileset, for this reason.

This set of restrictions feels properly absurd for today, but I've found it really somewhat manageable, and limitations do after all breed innovation.

The Tools

My primary deployment/testing platform is DOSBox-X, which I use to test my builds. It's a really decent DOS emulator that aims for maximal hardware compatibility and tries to make sure it can run a wide array of general-purpose DOS applications and not just games like several other DOSBox forks. It's stable, and it just so happens to emulate the exact damn processor my shitbox has, the Pentium III, down to the 866MHz clock. In retrospect, this was a very good pick, though at first I worried whether I could actually expect matching behaviour. The answer is a solid and resounding yes: everything I've run so far on DOSBox-X has run identically on the shitbox, down to the segfault traces being genuinely byte identical. Isn't emulation neat!

Now, I'm a C++ freak. The hammer I wield is C++, and everything looks like a nail. It's my go-to for scripting, it's my go-to for a Discord bot, it's what I use to solve anything that needs computation done. Thankfully, doing C++ on MS-DOS is genuinely easy and not even disgusting to think about. For this project, I naturally said hey, what if I did this in C++, and unexpectedly the DOS dev ecosystem has been pretty clean to work with. I am compiling code for a 32-bit MS-DOS target using a toolchain built off DJ Delorie's [3] DJGPP, containing GCC 15.2 with C++ support. This means that, yes, structured bindings and lambdas Just Work™ on MS-DOS targets, and that the MS-DOS has a newer compiler than my actual main machine running Linux Mint 22.3.

DJGPP includes Charles W. Sandmann's DOS extender CWSDPMI, which provides the system with virtual memory management and the ability to seamlessly run protected mode code on a real mode OS (such as MS-DOS is, or such as Win98 pretends to be). Memory protection, as annoying as it can be, is not a bad thing, and segfaulting on MS-DOS because you touched a nono array or did a double-free is great for debugging. [4]

DJGPP also includes a few more useful treats:

• a port of gdb for MS-DOS, running natively on the system (this is excellent)

• dynamic linker system (dxegen and the associated dxe3res)

• whole bunch of headers for useful operations (file.h and even a relatively performant regex.h)

It's a surprisingly robust system, and has been around for longer than I have. Did you know that DJGPP was what was used to compile Quake back in the day!?

Another good thing that DJGPP provides us with is the ability to have both a natively-running C/C++ compiler on the MS-DOS itself [5] and one that works on a modern 64-bit host. Cross-compiling for the MS-DOS is much, much, muuuuch faster than doing it natively (something on the order of 50x faster is what I've measured), and it enables me to do a modern type of incremental build, and also just edit while another thread is compiling the game. Lots of stuff you can do!

Finally, a very good cherry on top is that inline assembly and whole-cloth ASM subroutines can get integrated really easily into the codebase without much fanfare; I can just point the compiler at an .s file and it magically gets handled just the way I want it :) [6]

For the rest of the tech, I went with a trustworthy old friend: the game library Allegro, with one of the later versions (but not the very last one) of Allegro 4 still maintaining DOS support. This basically pushed most of the concerns out of my way, since Allegro would handle video mode toggling, input support (mouse and keyboard), timer interrupts (crucial for building an animation), and had support for a number of useful file formats like the .pcx image format. Allegro also provides good support for 16-bit colour [7] and even has hand-tuned assembly functions for drawing a large number of things (lines, circles, pixels) adjusted for 16-bpp colour mode. Allegro also has excellent compatibility with DJGPP.

Given that we're linking a large library such as Allegro into the final game executable, binary size is understandably a bit awful. The liballeg.a object is by itself 1495KB, which is already above what we can afford; aggressive LTO does handle this pretty well, but the resulting executable still ends up hovering at around 1.1MB, dangerously close to the limit, even when compiling with -Os and stripping as much as possible. This isn't a new problem, and developers of the time also encountered it frequently: the natural solution is to do what they did, and use an executable packer. Packers are programs that take an executable program, compress the majority of the code with some algorithm, and replace the entry point with a decompressor stub that would reconstruct external imports and decompress the packed executable and jump into it. Packers of yesteryear weren't quite as sophisticated as what we have today, but upx is pretty fresh, very good, and (most importantly) can work on MS-DOS executables of various formats, including the .EXEs that DJGPP produces. It claims to give up to 50% reduction in most cases; currently, I'm going from 1127KB to 321KB, which is ~28.5% of the initial size.

Development process

Honestly, I expected development to be much more painful than it really is.

My actual tech stack is, including some of the pieces I mentioned above, the following:

• DJGPP toolchain - GCC, G++ 15.2 running on a 64-bit Linux host - native gdb running on MS-DOS host

• upx packer

• stb_img for format support

• make to call compiler and packer

• Aseprite for graphics

• VS Code as the IDE (I like having popups telling me what a function wants)

• DOSBox-X for testing

How does the code look?

Like honest to God normal C++ code. If you don't believe me, just take a look at this:

void planet_t::update_minimap()
{
    if(minimap == nullptr) { return; }
    clear_bitmap(minimap);
    for(uint16_t j = 0; j < PLANET_ROWS; j++)
        for(uint16_t i = 0; i < PLANET_COLS; i++)
        {
            uint16_t colour = 0;
            switch(map[j][i].tileid)
            {
                _CASE(EMTY, EMTYCOL);
                _CASE(WATR, WATRCOL);
                _CASE(GRAS, GRASCOL);
                _CASE(SVNA, SVNACOL);
                _CASE(DEEP, DEEPCOL);
                _CASE(SNOW, SNOWCOL);
                _CASE(MEAT, MEATCOL);
                _CASE(SAND, SANDCOL);
                _CASE(VOID, VOIDCOL);
                _CASE(TOP1, TOP1COL);
                _CASE(TOP2, TOP2COL);
                _CASE(ROCK, ROCKCOL);
                _CASE(MNTN, MNTNCOL);
                _CASE(SWMP, SWMPCOL);

                default:
                    colour = makecol16(0xff, 0x00, 0x00);
                    break;
            }
            uint8_t x = (i * 3);
            uint8_t y = (j * 3);
            if((j % 2) == 0) { x += 1; }
            rectfill(minimap, x, y, x+2, y+2, colour);
        }
    return;
}

You can just do regular idiomatic C++, there's not a lot of gotchas. You don't have everything you would want on a modern system, so things like multithreading aren't in by default (though there exist POSIX thread library ports to DJGPP if you want them for some reason), but most things work just fine. Structured bindings are in, various deductions work fine, constructors and destructors are okay, exceptions work without much of a fuss, etc. With the toolchain it's less inconvenient to do dev for DOS than it is for Windows it feels like.

What's next?

Now, I'm actually a bit along the project, though it's still in the early stages; I work on it whenever I'm feeling particularly bored and there's nothing else to distract me. As I'm loading some configurations from disk, such as the terrain layouts of planets which are constructed from a bitmap/tilemap stored in the assets folder, I've also made some devtools to help me visualise these and work on them 'in-engine', like a micro level editor. I'll probably expand these into a bundle of more proper editing tools that I'll distribute to the end users, to allow them to make their own planets and maybe even events (if somewhen in the far future I get to making an event or scripting system...)

Currently, I have basic screen rendering done, tile mouseover detection is there (it's a bit messy figuring out which tile your mouse is over, actually), a minimap system is there, as is scrolling the planet map. Planets are for now cylindrical: they wrap around going east to west, but are capped off at the poles, and all latitudinal parallels are of equal length. I don't think I'll actually approach making a realistically spherical planet; spheres translate poorly to maps, and I'm a big fan primarily of maps.

If I come back to this devblog series, I'll be writing first about how I built the animation system, how I handled hex coordinates and printing hex tiles, etc. and show some actual screens of the game. They're still horrible.

Lay of the Land

I won't bore the audience with an elabourate description of the chip. There are eight general-purpose registers, a shadow registry for quick context switching in kernel, a hardware stack pointer, and a 24-bit address space. There is an MMU which pages out the memory space into 12-bit pages and tracks read-write permissions for each. None of that is especially important.

I decided I wanted to port a C compiler to target my chip so I could feel the transient emotional satisfaction of actually compiling a Thing of code for my chip. It's a form of validation: look at it! There it goes, there it works!

But porting a C compiler, or writing a new backend for one is, to put it lightly, a bit of a bother. Some projects are said to be easy to port, needing only a month or two of focused work to emit working code.

I'll show you how to do it in a day.

Our compiler

There are many C compilers out there, certainly dozens and probably reaching into the hundreds. Unlike writing a C++ compiler, which is a sisyphean task that even giants like Microsoft end up having trouble with, you can just sit down and crank out a working (albeit not good) C compiler and be the same age you were when you started it.

Most of them are not meant to be retargetable and so will produce code for only one platform, or will have a very small set of alternatives that are difficult to extend. The one C compiler I've found isn't that hard to work with is the excellent 8cc/ELVM pair. The 8cc compiler serves as the frontend and accepts normal-looking C code (including, surprisingly, itself), and emits ELIR assembly, while the elc backend consumes that and emits various other languages.

The code quality is atrocious. The ELIR architecture has four general-purpose registers and two special ones, only addition and subtraction for number manipulation (so not even bitwise operations), a few conditionals, and char input and output. The compiler is laughably stupid, producing code that would make gcc -O0 feel performant. That's fine, because we're looking for something that is easy to get working.

In the /target folder, you should create a file that will contain your entire backend. I went with enn.c, as the target is the enncpu. At the top you'll want to include the following:

...
#include <ir/ir.h>
#include <target/util.h>
...

These contain some of the basic definitions and things the compiler internals use.

The main datatypes you'll be dealing with are:

typedef enum {
  A, B, C, D, BP, SP
} Reg;

typedef enum {
  REG, IMM
} ValueType;

typedef enum {
  OP_UNSET = -2, OP_ERR = -1,
  MOV = 0, ADD, SUB, LOAD, STORE, PUTC, GETC, EXIT,
  JEQ = 8, JNE, JLT, JGT, JLE, JGE, JMP,
  // Optional operations follow.
  EQ = 16, NE, LT, GT, LE, GE, DUMP,
  LAST_OP
} Op;

typedef struct {
  ValueType type;
  union {
    Reg reg;
    int imm;
    void* tmp;
  };
} Value;

typedef struct Inst_ {
  Op op;
  Value dst;
  Value src;
  Value jmp;
  int pc;
  int lineno;
  char* magic_comment;
  struct Inst_* next;
} Inst;

typedef struct Data_ {
  int v;
  struct Data_* next;
} Data;

typedef struct {
  Inst* text;
  Data* data;
} Module;

The backend driver calls your target through one function, which should return void and take one pointer of type Module*. I named mine void target_enn(Module* module). The single argument is what the driver passes to your target and it contains the entirety of the information extracted from the code. This target is going to be pasted verbatim into the backend driver code and compiled into it, so it has to include a trailing newline (to avoid messing up includes that come after it; yes, I got burnt on this myself).

The ELIR virtual machine is a Harvard-architecture thing, and instruction and data codepaths do not mix. Instructions are not addressable, and all addressing is for the data. The compiler works with 24-bit words, and data is word-addressed. The compiler makes some convenient but wild assumptions (sizeof(char) == sizeof(int)), and this makes it both a bit easier and a bit more complicated to keep track of things.

The dataword stream and the instruction stream are both in the form of a linked list. The module provided by the caller contains pointers to the first members of both of those.

Instructions are the more fun part. Since they aren't actually addressable, the ELVM uses a genuinely horrendous scheme to keep track of where jumps go. Every piece of code under a label has the same pc address (so a sequence of a hundred instructions will all have pc == 34 if they are under the 34th label in the instruction stream!), and jumps go to the first instruction in that stream. It is best to tackle addressing at the end.

Before you implement the instructions, read up on their behaviour and whatnot; it's a very short read, and answers basically everything you need to know.

Emitting an instruction is fairly easy. The Inst struct (see above for its layout) has everything you need to build an instruction out of it. The way I handled it was:

static void enn_inst(Inst* inst)
{
        switch(inst->op)
        {
                case MOV:
                        if (inst->src.type == REG) {
                                emit_enn_mov_reg(inst->dst.reg, inst->src.reg);
                        } else {
                                emit_enn_mov_imm(inst->dst.reg, inst->src.imm);
                        }
                        break;
                case ADD:
                        if (inst->src.type == REG) {
                                emit_enn_add_reg(inst->dst.reg, inst->src.reg);
                        } else {
                                emit_enn_add_imm(inst->dst.reg, inst->src.imm);
                        }
                        break;
        // ...

static void emit_enn_add_reg(Reg dst, Reg src)
{ printf("\tADD   %c, %c\n", dst + 'A', src + 'A'); }

        // ...

Really, that's about all there is to it to emitting instructions. Keep in mind that immediates are ∈ [0, 16777215], and adjust accordingly.

Since labels/pc breaks don't actually appear in the stream but are signified by a change in the inst->pc value, you'll have to do a walk through the linked list to find out where these breaks lie. To avoid calculating the byte addresses of where those jumps go (as was done e.g. in the Armv7 and x86 backends), I exploit the fact that I'm generating assembly and offload the bookkeeping to the assembler. I do one walk through the instruction stream and just emit a string label whenever a break is encountered. Here is the whole of my target callback code:

void target_enn(Module* module)
{
        emit_enn_data_head();

        for(Data* data = module->data; data; data = data->next) {
                printf(".INT24 %d\n", data->v);
        }
        emit_enn_data_tail();

        emit_enn_head();

        int prev_pc = -1;

        for (Inst *inst = module->text; inst; inst = inst->next) {
                if (prev_pc != inst->pc) {
                        printf("@L%d\n", inst->pc);
                }
                enn_inst(inst);
                prev_pc = inst->pc;
        }

        emit_enn_tail();

        return;
} // must have tailing newline

Accordingly, when a jump is encountered, I load the address of the label and jump to it. Here is that bit:

static void emit_enn_jump_reg(Reg jmp)
{ printf("\tJMR   %c\n", jmp + 'A'); }

static void emit_enn_adrl_lbl(int jmp)
{
        printf("\tADRL  G, @L%d\n", jmp);
        printf("\tADRM  G, @L%d\n", jmp);
        printf("\tADRH  G, @L%d\n", jmp);
}

static void emit_enn_jump_imm(int jmp)
{
        emit_enn_adrl_lbl(jmp);
        emit_enn_jump_reg(6);
}

Data loads and stores are a bit more complicated. The ELIR abstraction starts placing the data at address 0, growing upwards, and each word is 3 bytes wide but word-addressed. This means that you need to start placing your data and memory at a known base address and calculate offsets off that. I did it brutishly: data is placed starting at 0x10, after the header, and then all addresses are adjusted based off that. Here is an example:

static void emit_enn_load_reg(Reg dst, Reg src)
{
        printf("\tMOV   G, #3\n");
        printf("\tMULA  %c, G\n", src + 'A');
        printf("\tADD   %c, #16\n", src + 'A');
        printf("\tLDRS  %c, %c\n", dst + 'A', src + 'A');
        printf("\tSUB   %c, #16\n", src + 'A');
        printf("\tDIV   %c, G\n", src + 'A');
}

static void emit_enn_load_imm(Reg dst, int src)
{
        src += 16;
        src *= 3;
        emit_enn_mov_imm(6, src);
        emit_enn_load_reg(dst, 6);
        emit_enn_sub_reg(6, 6);
}

In my case, I know that my base is at #16, and since I set up all the data as .INT24, everything is offset in multiples of three. While I considered padding things out between those to allow for LSHL G, #2 instead of the more convoluted multiplication, it would've eaten up somewhat more space in the data segment, so I decided against it (but will try it like that some time in the future).

Putting it together

To actually have your thing be usable, you have to do a small amount of further bookkeeping.

Your first stop will be the target/elc.c file, where you'll add your target callback. You really can't miss the huge block of function pointer declarations in there, trust me.

The second stop will be going into the Makefile and rummaging around. The ELVM suite is annoying to build because it runs a bunch of testsand samples on every invocation of make. You can change that, and I did, but I won't waste time here on the how since it's immaterial. In the Makefile you'll want to find the ELC_SRCS variable and append your backend target source file to it. After that, you'll want to go to the lists of targets under the comment line # Targets, and add yours. Here's mine:

TARGET := enn
RUNNER := tools/runenn.sh
include target.mk

The tools/runenn.sh file is just an empty bash file because I don't want it running any of those tests. You should then of course hit make -j 8 (or more; and there are a lot of noisy files in there so the console will start sparking and sizzling), and inevitably hit an error because the compilation is passed the unfortunate flags -W -Wall -Wextra -Werror so even an unused variable will break the build and you'll have to do it all over again and so on and so forth.

After this whole song and dance is finished, you'll be left with out/8cc and out/elc. The first is the C compiler frontend which takes one or more C files (there's a linker option that I didn't explore; so one in my case) and produces assembly out of that. You'll invoke it with 8cc {infile}.c -S and it will produce the corresponding {infile}.s (same name) in the same directory (I'm not sure yet how to get it to go anywhere else, but that's the least important thing here). After that, you will invoke the backend driver with elc {infile}.s -enn > outfile.txt (or wherever else you do output; I did mine to stdout). This should, in theory, leave you with a fully working Thing of assembly that targets your custom CPU (in practice: I had to fiddle with it a bit to get it going, and this revealed some bugs in the assembler I didn't know were there). Isn't that neat!

Outro

Realistically, it should take a decent C programmer about a day to actually write a working backend for 8cc using ELVM. The code is absolute gore, and there is so much room for optimisation (such as eliminating dead assignments), but this simplicity and lack of optimisation (though sorely needed) is probably a direct factor in how portable this thing really is. It took me about six hours and the end result was around 400 lines of surprisingly stupid C; some of those hours were wasted on figuring out some un(der)documented things using the x86 and Armv7 backends as reference.

Go do it yourself! Have a go at it, have some fun. I really doubt there's any other C compiler that's this trivial to port and can compile itself (seriously, that's one of the test cases and why it takes so goddamn long to make them).

Under this line you'll find the actual full source of my backend. It's dirty, and I slapped it together in one day. The version here isn't 100% identical to my own code as I shuffled around some newlines and whitespace to make it take up just slightly less space, but the contents are unchanged. Enjoy.

static void emit_enn_add_imm(Reg dst, int src);
static void emit_enn_sub_imm(Reg dst, int src);

static void emit_enn_head()
{
        printf(".SEC %%main\n");
        printf("\tADRL H, @_sec_main\n\tADRM H, @_sec_main\n\tADRH H, @_sec_main\n");
        printf("\tMOVL G, #0x80\n\tLSHL G, #2\n");
        printf("\tADD  H, G\n\tWSP  H\n\tSUB  G, G\n\n");
}
static void emit_enn_tail()
{ printf("%%main\n"); }

static void emit_enn_mov_reg(Reg dst, Reg src)
{ printf("\tMOV   %c, %c\n", dst + 'A', src + 'A'); }

static void emit_enn_mov_imm(Reg dst, int src)
{
        bool neg = false;
        if(src < 0) { src = -src; neg = true; }
        printf("\tMOVL  %c, #0x%02x\n",     dst + 'A', (byte)(src >>  0));
        if(src > 0xff)
                printf("\tMOVM  %c, #0x%02x\n", dst + 'A', (byte)(src >>  8));
        if(src > 0xffff)
                printf("\tMOVH  %c, #0x%02x\n", dst + 'A', (byte)(src >> 16));
        if(neg)
                printf("\tSUB   G, G\n\tSUB   G, %c\n\tMOV   %c, G\n", dst + 'A', dst + 'A');
}

static void emit_enn_add_reg(Reg dst, Reg src)
{ printf("\tADD   %c, %c\n", dst + 'A', src + 'A'); }

static void emit_enn_add_imm(Reg dst, int src)
{
        if(src < 0) { emit_enn_sub_imm(dst, -1 * src); return; }
        else if (src < 64)
                printf("\tADD   %c, #%d\n", dst + 'A', src);
        else
        {
                emit_enn_mov_imm(6, src);
                printf("\tADD   %c, G\n", dst + 'A');
        }
}

static void emit_enn_sub_reg(Reg dst, Reg src)
{ printf("\tSUB   %c, %c\n", dst + 'A', src + 'A'); }

static void emit_enn_sub_imm(Reg dst, int src)
{
        if(src < 0) { emit_enn_add_imm(dst, -1 * src); return; }
        else if (src < 64)
                printf("\tSUB   %c, #%d\n", dst + 'A', src);
        else
        {
                emit_enn_mov_imm(6, src);
                printf("\tSUB   %c, G\n", dst + 'A');
        }
}

static void emit_enn_load_reg(Reg dst, Reg src)
{
        printf("\tMOV   G, #3\n");
        printf("\tMULA  %c, G\n", src + 'A');
        printf("\tADD   %c, #16\n", src + 'A');
        printf("\tLDRS  %c, %c\n", dst + 'A', src + 'A');
        printf("\tSUB   %c, #16\n", src + 'A');
        printf("\tDIV   %c, G\n", src + 'A');
}

static void emit_enn_load_imm(Reg dst, int src)
{
        src += 16;
        src *= 3;
        emit_enn_mov_imm(6, src);
        emit_enn_load_reg(dst, 6);
        emit_enn_sub_reg(6, 6);
}

static void emit_enn_store_reg(Reg dst, Reg src)
{
        printf("\tMOV   G, #3\n");
        printf("\tMULA  %c, G\n", src + 'A');
        printf("\tADD   %c, #16\n", src + 'A');
        printf("\tSTRS  %c, %c\n", dst + 'A', src + 'A');
        printf("\tSUB   %c, #16\n", src + 'A');
        printf("\tDIV   %c, G\n", src + 'A');
}

static void emit_enn_store_imm(Reg dst, int src)
{
        src += 16;
        src *= 3;
        emit_enn_mov_imm(6, src);
        emit_enn_store_reg(dst, 6);
        emit_enn_sub_reg(6, 6);
}

static void emit_enn_exit()
{ printf("\tERR ; \n"); }

typedef enum {
        CEQ, CNE,
        CLT, CGT,
        CLE, CGE,
        NONE
} COMPARISON;

static void emit_enn_cmp_reg(COMPARISON type, Reg dst, Reg src)
{
        switch(type)
        {
                case CEQ: printf("\tCEQ   %c, %c\n", dst + 'A', src + 'A'); break;
                case CNE: printf("\tCNE   %c, %c\n", dst + 'A', src + 'A'); break;
                case CLT: printf("\tCLT   %c, %c\n", dst + 'A', src + 'A'); break;
                case CGT: printf("\tCGT   %c, %c\n", dst + 'A', src + 'A'); break;
                case CLE: printf("\tCGT   %c, %c\n\tCINV\n", dst + 'A', src + 'A'); break;
                case CGE: printf("\tCLT   %c, %c\n\tCINV\n", dst + 'A', src + 'A'); break;
                case NONE: return;
        }
}
static void emit_enn_cmp_imm(COMPARISON type, Reg dst, int src)
{
        if(type == NONE) { return; }
        if (src < 64)
                printf("\tMOVL  G, #%d\n", src);
        else
        {
                emit_enn_mov_imm(6, src);
                printf("\tADD   %c, G\n", dst + 'A');
        }
        switch(type)
        {
                case CEQ: printf("\tCEQ   %c, G\n", dst + 'A'); break;
                case CNE: printf("\tCNE   %c, G\n", dst + 'A'); break;
                case CLT: printf("\tCLT   %c, G\n", dst + 'A'); break;
                case CGT: printf("\tCGT   %c, G\n", dst + 'A'); break;
                case CLE: printf("\tCGT   %c, G\n\tCINV\n", dst + 'A'); break;
                case CGE: printf("\tCLT   %c, G\n\tCINV\n", dst + 'A'); break;
                case NONE: return;
        }
}

static void emit_enn_jmcc_reg(Reg jmp)
{ printf("\tJMR.P %c\n", jmp + 'A'); }
static void emit_enn_jump_reg(Reg jmp)
{ printf("\tJMR   %c\n", jmp + 'A'); }
static void emit_enn_adrl_lbl(int jmp)
{
        printf("\tADRL  G, @L%d\n", jmp);
        printf("\tADRM  G, @L%d\n", jmp);
        printf("\tADRH  G, @L%d\n", jmp);
}

static void emit_enn_jmcc_imm(int jmp)
{ emit_enn_adrl_lbl(jmp); emit_enn_jmcc_reg(6); }
static void emit_enn_jump_imm(int jmp)
{ emit_enn_adrl_lbl(jmp); emit_enn_jump_reg(6); }

static void emit_enn_putc(Reg reg)
{ printf("\tDBGC  %c\n", reg + 'A'); }

static void enn_inst(Inst* inst)
{
        switch(inst->op)
        {
                case MOV:
                        if (inst->src.type == REG) {
                                emit_enn_mov_reg(inst->dst.reg, inst->src.reg);
                        } else {
                                emit_enn_mov_imm(inst->dst.reg, inst->src.imm);
                        }
                        break;
                case ADD:
                        if (inst->src.type == REG) {
                                emit_enn_add_reg(inst->dst.reg, inst->src.reg);
                        } else {
                                emit_enn_add_imm(inst->dst.reg, inst->src.imm);
                        }
                        break;
                case SUB:
                        if (inst->src.type == REG) {
                                emit_enn_sub_reg(inst->dst.reg, inst->src.reg);
                        } else {
                                emit_enn_sub_imm(inst->dst.reg, inst->src.imm);
                        }
                        break;
                case EXIT:
                        emit_enn_exit();
                        break;
                case LOAD:
                        if (inst->src.type == REG) {
                                emit_enn_load_reg(inst->dst.reg, inst->src.reg);
                        } else {
                                emit_enn_load_imm(inst->dst.reg, inst->src.imm);
                        }
                        break;
                case STORE:
                        if (inst->src.type == REG) {
                                emit_enn_store_reg(inst->dst.reg, inst->src.reg);
                        } else {
                                emit_enn_store_imm(inst->dst.reg, inst->src.imm);
                        }
                        break;
                case JMP:
                        if (inst->jmp.type == REG) {
                                emit_enn_jump_reg(inst->jmp.reg);
                        } else {
                                emit_enn_jump_imm(inst->jmp.imm);
                        }
                        break;
                case PUTC:
                        emit_enn_putc(inst->src.reg);
                        break;
                case JEQ:
                        if (inst->jmp.type == REG) {
                                emit_enn_cmp_reg(CEQ, inst->dst.reg, inst->src.reg);
                                emit_enn_jmcc_reg(inst->jmp.reg);
                        } else {
                                emit_enn_cmp_imm(CEQ, inst->dst.reg, inst->src.imm);
                                emit_enn_jmcc_imm(inst->jmp.imm);
                        }
                        break;
                case EQ:
                        if(inst->src.type == REG)
                                emit_enn_cmp_reg(CEQ, inst->dst.reg, inst->src.reg);
                        else
                                emit_enn_cmp_imm(CEQ, inst->dst.reg, inst->src.imm);
                        break;
                case JNE:
                        if (inst->jmp.type == REG) {
                                emit_enn_cmp_reg(CNE, inst->dst.reg, inst->src.reg);
                                emit_enn_jmcc_reg(inst->jmp.reg);
                        } else {
                                emit_enn_cmp_imm(CNE, inst->dst.reg, inst->src.imm);
                                emit_enn_jmcc_imm(inst->jmp.imm);
                        }
                        break;
                case NE:
                        if(inst->src.type == REG)
                                emit_enn_cmp_reg(CNE, inst->dst.reg, inst->src.reg);
                        else
                                emit_enn_cmp_imm(CNE, inst->dst.reg, inst->src.imm);
                        break;
                case JLT:
                        if (inst->jmp.type == REG) {
                                emit_enn_cmp_reg(CLT, inst->dst.reg, inst->src.reg);
                                emit_enn_jmcc_reg(inst->jmp.reg);
                        } else {
                                emit_enn_cmp_imm(CLT, inst->dst.reg, inst->src.imm);
                                emit_enn_jmcc_imm(inst->jmp.imm);
                        }
                        break;
                case LT:
                        if(inst->src.type == REG)
                                emit_enn_cmp_reg(CLT, inst->dst.reg, inst->src.reg);
                        else
                                emit_enn_cmp_imm(CLT, inst->dst.reg, inst->src.imm);
                        break;
                case JGT:
                        if (inst->jmp.type == REG) {
                                emit_enn_cmp_reg(CGT, inst->dst.reg, inst->src.reg);
                                emit_enn_jmcc_reg(inst->jmp.reg);
                        } else {
                                emit_enn_cmp_imm(CGT, inst->dst.reg, inst->src.imm);
                                emit_enn_jmcc_imm(inst->jmp.imm);
                        }
                        break;
                case GT:
                        if(inst->src.type == REG)
                                emit_enn_cmp_reg(CGT, inst->dst.reg, inst->src.reg);
                        else
                                emit_enn_cmp_imm(CGT, inst->dst.reg, inst->src.imm);
                        break;
                case JLE:
                        if (inst->jmp.type == REG) {
                                emit_enn_cmp_reg(CLE, inst->dst.reg, inst->src.reg);
                                emit_enn_jmcc_reg(inst->jmp.reg);
                        } else {
                                emit_enn_cmp_imm(CLE, inst->dst.reg, inst->src.imm);
                                emit_enn_jmcc_imm(inst->jmp.imm);
                        }
                        break;
                case LE:
                        if(inst->src.type == REG)
                                emit_enn_cmp_reg(CLE, inst->dst.reg, inst->src.reg);
                        else
                                emit_enn_cmp_imm(CLE, inst->dst.reg, inst->src.imm);
                        break;
                case JGE:
                        if (inst->jmp.type == REG) {
                                emit_enn_cmp_reg(CGE, inst->dst.reg, inst->src.reg);
                                emit_enn_jmcc_reg(inst->jmp.reg);
                        } else {
                                emit_enn_cmp_imm(CGE, inst->dst.reg, inst->src.imm);
                                emit_enn_jmcc_imm(inst->jmp.imm);
                        }
                        break;
                case GE:
                        if(inst->src.type == REG)
                                emit_enn_cmp_reg(CGE, inst->dst.reg, inst->src.reg);
                        else
                                emit_enn_cmp_imm(CGE, inst->dst.reg, inst->src.imm);
                        break;
                default: printf("something went wrong with insn %d (d:%d, s:%d, j:%d)\n",
                        inst->op, inst->dst.imm, inst->src.imm, inst->jmp.imm);
        }
}

static void emit_enn_data_head()
{
        printf("\n.SEC %%data\n");
        printf("\tADRL  G, @_sec_data\n");
        printf("\tADRM  G, @_sec_data\n");
        printf("\tADRH  G, @_sec_data\n");
        printf("\tJMR   G\n");
}
static void emit_enn_data_tail()
{ printf(".PAD\n%%data\n"); }

void target_enn(Module* module)
{
        emit_enn_data_head();
        for(Data* data = module->data; data; data = data->next)
                printf(".INT24 %d\n", data->v);

        emit_enn_data_tail();
        emit_enn_head();
        int prev_pc = -1;
        for (Inst *inst = module->text; inst; inst = inst->next) {
                if (prev_pc != inst->pc) { printf("@L%d\n", inst->pc); }
                enn_inst(inst);
                prev_pc = inst->pc;
        }
        emit_enn_tail();
        return;
} // must have tailing newline

Prelude

Recently I've been toying with some weird, cheap SBCs that offer a lot for way too little money. You can actually get a device with a pressure-sensitive 320x240 screen, 4M flash and 512K RAM for under ten bucks so obviously what I did was that I did get one of those and decided to see what I could do with it.

Enter the Cheap Yellow Display (or CYD for short), name courtesy of Brian Lough, that's got one of those fancy Espressiv IoT SOCs, with a Bluetooth stack and WiFi out of the box, Arduino integration, two cores (one of which will run the RTOS in an Arduino setup by default, but you can hijack it and run other code on it), easy interrupt registering (with per-core control to boot), a decent amount of GPIO pins broken out, and a resistive LCD touchscreen that's trivial to program for.

Its weakest points are its 520K RAM and 4M flash. This is probably the one thing that can limit its usability in general, and there are other ESP32 chips and boards with different configs (8M and 16M flash variants exist, and many boards also integrate an extra 4/8M of PSRAM, absent on the CYD), but this is already more than enough for a lot of things you'd want to do. If you're crazy enough, you can even run Linux on it (though the device in the link has 8M of PSRAM, you can probably trim it and make a godawful, pained, miserable Linux with 512K as well, and it boots just "fine"), but we're not going down this route because that just feels like hubris: yes, the old UNIXes did run on hardware with 64-256K RAM, but that was more than 50 years ago and those machines ate around 9 amps of current at 230V (or at least that's what I'm getting for the PDP-11/20 power supply); the CYD works at a steady 3.3V/115mA, and is a quintillion times faster [1] with a much saner work environment and tools that don't make you want to immediately [2] kill yourself, and you can replace it with basically infinite ease if one dies. Try replacing a PDP-11 on a budget.

Some good things about the lack of PSRAM, though, is that it teaches you to use what you have and not be a pussy ​ has much slower access times and as such you'll want to fall back on fast internal RAM anyhow; PSRAM would be good for memory you'd access intermittently rather than hit every cycle. Or maybe this is all just cope. In any case, 520K RAM it is.

It's also a bit more muddied than that: the ESP32 divides its memory space into two blocks, DRAM for data and IRAM for instructions. These are asymmetric: DRAM covers some 320K and IRAM covers 200K. The docs helpfully state that "[a]ny internal SRAM which is not used for Instruction RAM will be made available as DRAM (Data RAM) for static data and dynamic allocation (heap)", which means that if we need >328K data we can claw it back from IRAM, which helps our case a bit. The ability to execute things from RAM can also help with hot-loading code from external memory (I've looked a lot into this and concluded that just writing a parser for external code and then interpreting it is much, much less painful than hotloading code into IRAM, and we're currently in the middle of me musing how to implement that properly), so you can actually just drive a slim OS in RAM and load programs off an SD card, and one project does just that [3] with quite a bit of charm and polish to it.

The thing is, the ESP32 is powerful enough to run a RTOS (in this case it bundles FreeRTOS [4] by default and uses it for a bunch of shit), and this RTOS reserves some memory for itself, and the WiFi and BT stacks can also eat into this memory for their own purposes (this eats up a whole 64K of RAM in case you're wondering). A default setup with Arduino enabling the use of the Serial library without any screen code has the non-user code reserve 20080 bytes for global variables, "leaving 307600 bytes for local variables". The astute reader will see that this sums up to 327680B, i.e. 320K of RAM. A decent amount of this is going to the RTOS task scheduler and internal states, and a bit is going on facilitating dynamic memory allocation. The docs also tell us that there's a technical limitation to how much we can allocate in what way: we're limited to 160K static allocation, and the remaining 160K "can only be allocated at runtime as heap".

Loss to overhead isn't that bad, though, truth be told. A more full-featured programme with a screen and touch driver and SD card code for both reading and writing takes up 22560B, just under 2.5K more. This is not that awful considering how much RAM we have, but it'd be proper deadly on a smaller device like one of the actual Arduinos that have 2.5K RAM in total.

But this all also means that, if we want to use a lot of RAM in our thing, we have to dynamically allocate more when we start touching the >128K zone. Dynamic memory allocation on microcontrollers sucks, though, and you'll fragment the memory space in basically no time and won't be able to recover from it because there's no actual OS to bail you out. People die when an MCU upredictably returns nullptr from a malloc() call.

One of the solutions is to just allocate everything you can statically ahead of time, and then 'dynamically' allocate during startup and then never clean up or call free() again. The fact that malloc() can only give us blocks up to the size of the largest contiguous memory block and the fact that we're not doing any sophisticated memory mapping on the fly, this really just means figuring out how big our maximum memory blocks are and then allocating up to their size and then writing management code ourselves on top of that. Several tactics for this include memory pools and slab allocation, both of which are pretty neat ideas, and we'll be tapping into their concepts for our purposes. Let's dive in.

Diving In

There's something charming about the CYD. Like I've mentioned above, it's a dual-core system with 520K RAM and whatnot. The ISA the chips are running can be called unusual: it's called Xtensa and was developed by the IP company Tensilica. The exact architecture here is the Xtensa LX6 configured to use 32-bit transfers/data buses, and is little-endian. The CYD has a core without vector extensions, but there are Tensilica cores with SIMD (the ESP32-S3 is one such example, with some weird SIMD features that are genuine nice-to-haves). The chip offers some good features, a large number of pins broken out, several serials, JTAG, all the good stuff. The CYD cuts into this a decent bit by integrating an SD card module, multicoloured LED, a screen, a touch interface for that screen, a micro-USB port, and a buzzer/speaker two-wire/mono plug. The result is a still-decent package with 3 clusters of 4 pins exposed, one of which has a 3.3V line, and one of which doubles as a serial port.

These memory investigations are basically me building up some infrastructure to support an interpreted programming language running behind the scenes on the CYD, as a sort of native BASIC-like. There is no practical use to this, it's just me playing with my toys.

Given that the system has (at least) 320K of SRAM dedicated to data, this is obviously a good place to start. As no more than 160K can be statically allocated, and a decent amount of it will be eaten up by libraries and other code, it becomes "natural" to me to think about the CYD's RAM in terms of 64K blocks. This might be a problem [5] later, but currently it seems fine enough.

The screen itself will also want at least 75K of RAM (320 * 240 = 76800, so if every pixel is one byte that's a clean 75K; though colour performance might suffer obviously), and possibly more (the screen has 16-bit colour, so 150K RAM, eating up almost half of DRAM in one go). If I buffer the screen and expose it as a memory zone, that is; this isn't required in any way, and is anyway not something I care about right now.

In any case, it would be prudent to allocate as much memory as possible for the application, then manage it manually later, writing an own allocator and deallocator. It's also perfectly possible to write a kind of "memory mapper" that would provide a flat contiguous memory space to user code obfuscating the bullshit underneath. One pseudocode example of such a a remapper would be:

#include <vector>

struct memory
{
        std::array< std::array<uint8_t, 64 * 1024>, 4> contents;

        memory()
        {
                for(auto i : contents)
                        i = new std::array<uint8_t, 64 * 1024>;
        }
        ~memory() = delete; // must be static

        uint8_t read(uint32_t address)
        {
                uint8_t  page = (address >> 16) & 0xff;
                uint16_t cell =  address & 0xff'ff;

                return contents[page%4]->at(cell);
        }
        void write(uint32_t address, uint8_t payload)
        {
                uint8_t  page = (address >> 16) & 0xff;
                uint16_t cell =  address & 0xff'ff;

                contents[page%4]->at(cell) = payload;
        }
};

This kind of structure has sub-32 bytes of internal state, manages pages of 64K memory, and translates accesses in a perceived continuous space to accesses in a discontinuous space. Of course, this kind of code is unacceptable for embedded because using the STL in the first place is what is called "rough luck", but it's not the worst thing ever, and the code scaffold is good enough. This is downright primitive, but it serves the purpose of unifying a disparate memory space into one field. A lot more can be added on top of this to make it a proper manager unit that would provide virtual memory with memory safety features. This isn't currently too important.

This feels like the wrong solution for specific problems, though. Some other things I've toyed with include:

• Hash-maps

• Slabs and pools (aforementioned)

• Naïve constrained declaration and no dynamicity

• other detritus and miscellanea

The hash-map was probably my favourite idea, though.

// bias = 0.0049752954186705221
uint16_t hash1(uint16_t x)
{
    x ^= x >> 11;
    x *= 0x104bU;
    x ^= x >> 9;
    x *= 0xcd1bU;
    x ^= x >> 7;
    x *= 0x42a5U;
    x ^= x >> 11;
    x *= 0x258dU;
    x ^= x >> 9;
    return x;
}

// bias = 0.0046809801814303642
uint16_t hash2(uint16_t x)
{
    x ^= x >> 7;
    x *= 0xb257U;
    x ^= x >> 9;
    x *= 0x8829U;
    x ^= x >> 4;
    x *= 0x0af7U;
    x ^= x >> 12;
    x *= 0x504dU;
    x ^= x >> 6;
    return x;
}

uint16_t hash(uint32_t input)
{
        uint16_t a, b, c;
        a = hash1(input & 0xff'ff);
        b = hash2(input >> 16);
        c = hash1(a ^ b);

        return c;
}

bool hash_insert(uint32_t input)
{
        uint16_t a, b, c;
        a = hash1(input & 0xff'ff);
        b = hash2(input >> 16);
        c = hash1(a ^ b);

        if(!collides(c))
        {
                insert_at(c);
                return true;
        }
        else
        {
                c = hash2(c);
                if(!collides(c))
                {
                        insert_at(c);
                        return true;
                }
                else
                {
                        for(int i = 0; i < SIZE; i++)
                        {
                                c = c + 1;

                                if(!collides(c))
                                {
                                        insert_at(c);
                                        return true;
                                }
                        }
                        return false;
                }
        }
}

struct entry_t
{
        uint32_t name;          // 4
        uint32_t data;          // +4
};

static entry_t name_table[1024];

What else?

Really, it's just me staring at the screen blarting out words as they come, without a coherent idea of where it's going. I'm going to be designing the BASIC-like over the next few weeks/months (depending on how hard it holds my attention, really), and will probably be writing some kind of static hashmap (more sophisticated than the pseudocode above), a stack (possibly templated for type and depth), and a rudimentary array allocator that would implement a basic (hehe) heap. I'm not actually sure how useful multi-dimensional arrays were back in the olden days, though; I can imagine a 2D or 3D array doing just fine, but do you really need a 37D array when you're not doing some fancy linear algebra churning vectors (which you shouldn't be doing on an ESP32 anyway)?

Target Analysis

Just because we're bitchin programmers doesn't mean we can just write our code for all the processors out there and expect it to work: we're working strictly in Aarch64 land, and this means we have actually a very limited set of platforms we can ever dare to support without spinning up a whole new codebase and toolkit. Apart from that, we're coding for the virt board (or, well, "board", you know) that has its own quirks and peculiarities that other boards don't have, and in turn the other boards have their kinks to be ironed out (like the Pi actually booting from the GPU and not CPU).

Since we're writing and targetting Aarch64, the ARM CPU core designs we can think of are:

• ARMv8.1-A: A34, A35, A53, A57, A72, A73

• ARMv8.2-A: A55, A65, A75, A76, A77, A78, X1, Neoverse N1

• ARMv8.4-A: A65AE (partial), V1

• ARMv9.0-A: A510, A710, X2, Neoverse N2

QEMU gives us a number of other options, of which only the Fujitsu A64FX core has any merit—it's an ARMv8.2-A core with the 512-bit SVE extension.

There's also the option of using the QEMU max CPU—a pseudo-core that has the maximum featureset possible implemented in QEMU's codebase. This does not correspond to any core, and has a number of extensions that aren't in any of the currently emulated cores. For example, while the Fujitsu core is emulated with 512-bit SVE, you can set your max core to work with 1024-bit SVE registers 'out of the box'.

The QEMU codebase is, of course, incomplete. Some things are just not emulated yet, but you can get close enough. For example, there's no support for Apple chips, but they're running ARMv8.5-A; there's also no support for ARMv9.0-A cores but their features are more-less completely implemented and we're just waiting for an actual core such as the A510 to be written.

If we are going to write truly board-independent Aarch64 code, that means we're going to have to write peripheral detection routines and whatnot, and that's also a pain in the ass to do compared to just reading the device tree and pointing the CPU in the right direction, so we're avoiding that for now as well. The UART on a virt is in a totally different place compared to the two UARTs on a Pi3, for example, though the setup code is much the same.

But even within our virt garden there's a number of things that can change based on what core we're using. Today we're gonna take a look at two such features, see how they're implemented or left unimplemented, and handle them in two different ways.

Random Numbers

The first feature I wanna take a look at is the hardware RNG that some Arm cores have and some don't. There's a processor feature that determines whether there's a RNG accessible through mrs instructions, and that's FEAT_RNG that's expressed in the top four bits of the ID_AA64ISAR0_EL1 register.

Since we're gonna write a function that returns random numbers on the stack, we're gonna want to set up a check in the bootloader to see whether we have this feature enabled, and if not to rewrite the function address to the alternative code path.

The subroutine which we'll rely on to get random numbers from is going to be called _rng_64, and we'll use this same alias for both the hardware RNG and our fallback implementation.

Somewhere in our code, we're gonna add the following stub:

_rng_64_branch: .quad _rng_64_fallback
_rng_64:
        sub sp, sp, 8
        str x0, [sp, -8]!
        adr x0, =_rng_64_branch
        ldr x0, [x0]
        br  x0
_rng_64_hardware:
        ldr x0, [sp], 8
        ret
_rng_64_fallback:
        ldr x0, [sp], 8
        ret

We're going to use this stub as our jump location, and use the boot code to change the fallback branch to the hardware RNG branch if we have the RNG enabled.

In the boot sequence we'll then add this:

mrs x0, ID_AA64ISAR0_EL1
lsr x0, x0, 60
and x0, x0, 1

cbz x0, _skip_rng_fallback

adr x0, _rng_64_branch
adr x1, _rng_64_hardware
str x1, [x0]

_skip_rng_fallback:

We're overwriting the pointer if the hardwarve feature is enabled, since its default value is _rng_64_fallback. The hardware RNG is then used like this:

_rng_64_hardware:
 mrs x0, s3_3_c2_c4_0           // rndr
 str x0, [sp, 8]
 ldr x0, [sp], 8
 ret

Since we're using the same convention as with other functions, branching into the subroutine using the bl instruction, returning arguments on the stack, here we read the hardware register into x0, store it on the preallocated stack space, restore the x0 register and ret.

the memory map is as follows:

        0x40000000        -- QEMU virt ram and kernel start here
        STK               -- depending on the size of the kernel
                             the bottom of the stack can move around
        STK + 0x00100000  -- top of stack (1mb space), bottom of vectors
        STK + 0x00101000  -- top of vectors, bottom of ramfb config
        STK + 0x00101080  -- top of ramfb config (128b), bottom of ramfb itself
        STK + 0x00501080  -- top of ramfb (4mb space), bottom of heap
        STK + 0x10501080  -- top of heap

Keep in mind that the entire kernel and all memory above it are all rewritable code, and thus we must handle everything with care in order to not rewrite ourselves. We are still working with a flat memory model, and there are no MMU-based safety features; the code is fully privileged since we're still operating in kernel mode. The framebuffer device is basically just a chunk of RAM we dedicate that QEMU will read out of and blit to the screen. Framebuffers are the simplest model of graphical output you can get on any modern hardware, and they're also the least powerful and tend to be performance suckers. To configure the framebuffer, we must first communicate with QEMU itself, by trawling the QEMU config tree in the device's memory, figuring out if we can use the DMA interface, then writing the configuration through DMA to QEMU's config in big-endian order (!!), and only then can we use the framebuffer. Note that the framebuffer requires fw_cfg to have an enabled DMA, so our first step would be to check if DMA is enabled and, if not, to skip framebuffer setup completely and only work with UART down the line. see: https://github.com/qemu/qemu/blob/e93ded1bf6c94ab95015b33e188bc8b0b0c32670/hw/display/ramfb.c#L124 We'll do this by setting a flag if the DMA is not available, and later if we write applications that require the framebuffer we'll check whether it's enabled and gracefully exit informing the user that the functionality isn't available instead of writing to memory that doesn't have any visible effect.

UART revisited

The UART interface, like we covered, is dead simple. As with our output code, our input code is going to be fully blocking on the CPU until the UART flags change. Unlike the output, this means that we're going to be eating CPU time until the user provides with input, which could take a while. Cheap, nonblocking IO would take much more of a stack than we have going for us.

Our function will return a single character from input, and take no arguments. We will need to check if the RXFE flag is set (that is, that the receive stack is empty) on the UART, and wait until it's cleared.

Thankfully, the UART is buffered, so we don't need to time our reads from the data register to the exact moment the user writes something.

// our input function
// takes 0 arg on stack, returns 1
// trashes 2 registers
_ugetc:
        str  xzr, [sp, -8]!
        stp  x0, x1, [sp, -16]!

        adrp x0, UART_BASE
        add  x0, x0, :lo12:UART_BASE
        ldr  w0, [x0]
                        // now x0 has the UART_BASE location

        add  x0, x0, 0x18    // UART_FLAG address

        _ugetc_loop1:
                ldr  x1, [x0]
                and  x1, x1, 0x10
                cbnz x1, _ugetc_loop1

        sub  x0, x0, 0x18
        ldr  x0, [x0]
        str  x0, [sp, 16]
        ldp  x0, x1, [sp, 16]!
        ret

We reserved a place on the stack for the return argument, then stored the return there, while popping everything else.

Some fun code to play around with in the main code path:

PLEASE_WRITE:   .asciz "Please input a key and I'll do my best to repeat it and tell you if it's odd or even: "

3:
        ldr x2, =EXAMPLE_STRING
        str x2, [sp, -8]!
        bl _uputs
                // and Hello World, finally!

        ldr x2, =PLEASE_WRITE
        str x2, [sp, -8]!
        bl _uputs

        _parity_loop:
                bl  _ugetc
                ldr x2, [sp]
                and x2, x2, 0xff

                sub x2, x2, 0x40
                cbz x2, 4f
                add x2, x2, 0x40

                bl  _uputc

                sub x2, x2, 0x30
                and x2, x2, 0x1
                cbz x2, _is_even

                        _is_odd:
                        mov x2, 0x4f
                        b _parity_loop_end

                        _is_even:
                        mov x2, 0x45

                _parity_loop_end:
                str x2, [sp, -8]!
                bl _uputc
                b  _parity_loop

4:
        add x2, x2, 0x40
        str x2, [sp, -8]!
        bl _uputc
        b .

This program tests your input whether it's even or odd, prints E or O depending on the case, and halts when it encounters 0x40—that is, @.

To get the input working correctly, I had to pipe it to a serial device, and use PuTTY to link to the other side. If your input isn't working, it's probably due to that. My own experience here is that the code is somehow extremely sluggish; there's up to a full second of delay between my input and the device's response, and I'm not sure if that's a quirk of using virtual COM ports and PuTTY to interface with QEMU or a failing of my code. I might want to write some sort of device driver for input later on to alleviate that and avoid serial TTY, so we'll see.

Cleaning up

Last time I cut some corners in that I hardcoded the UART memory address but now we're gonna properly load it from a label that we can conveniently change if, somewhere down the line, the UART base changes or if something else happens that we gotta take care of. The start of label 1 will now be:

        UART_BASE: .word 0x09000000
//      UART_DATA: .byte 0x00
//      UART_FLAG: .byte 0x18
//      UART_CNTL: .byte 0x30
//      UART_FIFO: .byte 0x34
//      UART_INTC: .byte 0x44

        .section .text
        1:
                adrp x0, UART_BASE
                add  x0, x0, :lo12:UART_BASE
                ldr  w0, [x0]

                add  x0, x0, 0x30               // UART_CNTL

Not much has changed, except we're now loading the address. You'll note that it's a federal fucking issue to load from memory in Aarch64. We're first loading the page address of where the UART_BASE label is located into x0 via the adrp instruction; this loads the higher 56 bits of the address into the register, which then forces us to add the offset separately using add and the :lo12:(name) specifier, where the assembler helpfully extracts the lower 12 bits (hence the name) of the label for us and generates a normal add instruction with the offset as an immediate.

Two pits I fell into while playing with this:

1. sometimes the assembler fails to error on unaligned access and will joyfully read instructions one byte off

2. ldr x0, [x0] was a big headache since reading from a word into a dword register reads the word twice, once in low and once in high bits

The second one I keep forgetting about every now and then, I'm 100% sure I'll encounter it again. It's not mentioned in the official online docs, and I don't even see it mentioned in the 5200 (!) page architecture reference manual, though that might be my own search fuckup.

When you make and run this, it should do the same thing as before. We're not gonna load the offsets from memory because they're tiny (meaning it's much more of a chore to load them and add them to the base) [1] and, more importantly, they're guaranteed to never change relative to base and each other. We're also going to set UART up only once and won't change anything in the control register in the foreseeable future.

UART Safety, Printing and You!

The PL011 UART is equipped with a flag register that can tell us what the UART is doing right now and how it's going, so we can poll it to see whether it's ready to take a byte from us to print or not. From a quick look at the docs we see that we need to test for whether UARTFR & 0x28 returns true and wait until it isn't.

We'll also want to compartmentalise the print into a procedure or function. This is going to mean we'll have to think about how to pass arguments to functions, since our putchar() lookalike will consume a single byte that its caller will pass into it. Remember that spiel about calling conventions?

We'll call the routine _uputc, standing for "U(ART)-put-c(har)".

We can write the function out in high-level pseudocode to see exactly what's going to have to happen:

void _uputc(byte x)
{
        REGISTER  a;
        REGISTER* b = &UARTDR;

        b += 0x18;

        do
        {
                a = *b;
        } while (a & 0x28);

        b -= 0x18;

        *b = x;

        return;
}

This immediately tells us we need three registers to store the UART flags (a), the UART address (b), and the argument to send to the UART data register (x).

mrs x0, SCTLR_EL1
mov x1, 0x1a
neg x1, x1
and x0, x0, x1
msr SCTLR_EL1, x0

    ldr x0, =stack_top
    add x0, x0, 0x4
mov sp, x0

void _uputc(byte x)
{
        REGISTER  a;
        REGISTER* b = &UARTDR;

        b += 0x18;

        do
        {
                a = *b;
        } while (a & 0x28);

        b -= 0x18;

        *b = x;

        return;
}

mov x17, 0x48        // the character 'H'
str x17, [sp, -8]!   // our output argument
bl _uputc

// our print function
// takes 1 arg on stack, returns 0
// trashes 3 registers
_uputc:
        stp  x0, x1, [sp, -16]!
        str  x2, [sp, -8]!

        ldr  x2, [sp, 24]    // this is where we first pushed
                             // the argument in the caller

        adrp x0, UART_BASE
        add  x0, x0, :lo12:UART_BASE
        ldr  w0, [x0]
                        // now x0 has the UART_BASE location

        add  x0, x0, 0x18    // UART_FLAG address

        _uputc_loop1:
                ldr  x1, [x0]          // read from UART_FLAG
                and  x1, x1, 0xff      // 0010 1000 = busy & transmit full
                bic  x1, x1, 0xc0      // but we gotta do it the long way
                bic  x1, x1, 0x10
                bic  x1, x1, 0x07
        cbnz _uputc_loop1

        sub  x0, x0, 0x18      // back to BASE / DATA
        str  x2, [x0]

        ldr  x2, [sp], 8
        ldp  x0, x1, [sp], 16

        ret

// our print function
// takes 1 arg on stack, returns 0
// trashes 2 registers
_uputc:
        stp  x0, x1, [sp, -16]!

        adrp x0, UART_BASE
        add  x0, x0, :lo12:UART_BASE
        ldr  w0, [x0]
                        // now x0 has the UART_BASE location

        add  x0, x0, 0x18    // UART_FLAG address

        _uputc_loop1:
                ldr  x1, [x0]          // read from UART_FLAG
                and  x1, x1, 0xff      // 0010 1000 = busy & transmit full
                bic  x1, x1, 0xc0      // but we gotta do it the long way
                bic  x1, x1, 0x10
                bic  x1, x1, 0x07
        cbnz _uputc_loop1

        sub  x0, x0, 0x18      // back to BASE / DATA
        ldr  x1, [sp, 16]
        str  x1, [x0]

        ldp  x0, x1, [sp], 16
        add  sp, sp, 8         // pop the argument off the stack

        ret

Nested function calls

A peculiarity of the Aarch64 ISA is that the bl instruction lets us do branches to subroutines without any memory access, which is extremely good. The way it works is that it stores the address of the instruction it's accessing + 4 into the register x30, then jumping to the label we give it. Correspondingly, we return from the subroutine we jumped to using the ret instruction, which jumps to the address stored in x30 (i.e. it's an alias).

Predictably, this means that you can only bl exactly once before you trash your original return address.

This pair of instructions has its advantages over the x86 call and ret pair—no stack access is performed and things are kept fast, but you have to write the nesting code yourself and that gets kind of nasty kind of quickly.

We can thus absolutely avoid having to use the stack for subroutine calls that are one-deep, which saves on memory at the expense of losing a register.

The next function we'll write will have to call another function and so we'll have to write nesting code.