Tag Archives: tech

Old CSS, new CSS

Post Syndicated from Eevee original https://eev.ee/blog/2020/02/01/old-css-new-css/

I first got into web design/development in the late 90s, and only as I type this sentence do I realize how long ago that was.

And boy, it was horrendous. I mean, being able to make stuff and put it online where other people could see it was pretty slick, but we did not have very much to work with.

I’ve been taking for granted that most folks doing web stuff still remember those days, or at least the decade that followed, but I think that assumption might be a wee bit out of date. Some time ago I encountered a tweet marvelling at what we had to do without border-radius. I still remember waiting with bated breath for it to be unprefixed!

But then, I suspect I also know a number of folks who only tried web design in the old days, and assume nothing about it has changed since.

I’m here to tell all of you to get off my lawn. Here’s a history of CSS and web design, as I remember it.


(Please bear in mind that this post is a fine blend of memory and research, so I can’t guarantee any of it is actually correct, especially the bits about casuality. You may want to try the W3C’s history of CSS, which is considerably shorter, has a better chance of matching reality, and contains significantly less swearing.)

(Also, this would benefit greatly from more diagrams, but it took long enough just to write.)

The very early days

In the beginning, there was no CSS.

This was very bad.

My favorite artifact of this era is the book that taught me HTML: O’Reilly’s HTML: The Definitive Guide, published in several editions in the mid to late 90s. The book was indeed about HTML, with no mention of CSS at all. I don’t have it any more and can’t readily find screenshots online, but here’s a page from HTML & XHTML: The Definitive Guide, which seems to be a revision (I’ll get to XHTML later) with much the same style. Here, then, is the cutting-edge web design advice of 199X:

Screenshot of a plain website in IE, with plain black text on a white background with a simple image

Clearly delineate headers and footers with horizontal rules.

No, that’s not a border-top. That’s an <hr>. The page title is almost certainly centered with, well, <center>.

The page uses the default text color, background, and font. Partly because this is a guidebook introducing concepts one at a time; partly because the book was printed in black and white; and partly, I’m sure, because it reflected the reality that coloring anything was a huge pain in the ass.

Let’s say you wanted all your <h1>s to be red, across your entire site. You had to do this:

1
<H1><FONT COLOR=red>...</FONT></H1>

every single goddamn time. Hope you never decide to switch to blue!

Oh, and everyone wrote HTML tags in all caps. I don’t remember why we all thought that was a good idea. Maybe this was before syntax highlighting in text editors was very common (read: I was 12 and using Notepad), and uppercase tags were easier to distinguish from body text.

Keeping your site consistent was thus something of a nightmare. One solution was to simply not style anything, which a lot of folks did. This was nice, in some ways, since browsers let you change those defaults, so you could read the Web how you wanted.

A clever alternate solution, which I remember showing up in a lot of Geocities sites, was to simply give every page a completely different visual style. Fuck it, right? Just do whatever you want on each new page.

That trend was quite possibly the height of web design.

Damn, I miss those days. There were no big walled gardens, no Twitter or Facebook. If you had anything to say to anyone, you had to put together your own website. It was amazing. No one knew what they were doing; I’d wager that the vast majority of web designers at the time were clueless hobbyist tweens (like me) all copying from other clueless hobbyist tweens. Half the Web was fan portals about Animorphs, with inexplicable splash pages warning you that their site worked best if you had a 640×480 screen. (Any 12-year-old with insufficient resolution should, presumably, buy a new monitor with their allowance.) Everyone who was cool and in the know used Internet Explorer 3, the most advanced browser, but some losers still used Netscape Navigator so you had to put a “Best in IE” animated GIF on your splash page too.

This was also the era of “web-safe colors” — a palette of 216 colors, where every channel was one of 00, 33, 66, 99, cc, or ff — which existed because some people still had 256-color monitors! The things we take for granted now, like 24-bit color.

In fact, a lot of stuff we take for granted now was still a strange and untamed problem space. You want to have the same navigation on every page on your website? Okay, no problem: copy/paste it onto each page. When you update it, be sure to update every page — but most likely you’ll forget some, and your whole site will become an archaeological dig into itself, with strata of increasingly bitrotted pages.

Much easier was to use frames, meaning the browser window is split into a grid and a different page loads in each section… but then people would get confused if they landed on an individual page without the frames, as was common when coming from a search engine like AltaVista. (I can’t believe I’m explaining frames, but no one has used them since like 2001. You know iframes? The “i” is for inline, to distinguish them from regular frames, which take up the entire viewport.)

PHP wasn’t even called that yet, and nobody had heard of it. This weird “Perl” and “CGI” thing was really strange and hard to understand, and it didn’t work on your own computer, and the errors were hard to find and diagnose, and anyway Geocities didn’t support it. If you were really lucky and smart, your web host used Apache, and you could use its “server side include” syntax to do something like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
<BODY>
    <TABLE WIDTH=100% BORDER=0 CELLSPACING=8 CELLPADDING=0>
        <TR>
            <TD COLSPAN=2>
                <!--#include virtual="/header.html" --> 
            </TD>
        </TR>
        <TR>
            <TD WIDTH=20%>
                <!--#include virtual="/navigation.html" --> 
            </TD>
            <TD>
                (actual page content goes here)
            </TD>
        </TR>
    </TABLE>
</BODY>

Mwah. Beautiful. Apache would see the special comments, paste in the contents of the referenced files, and you’re off to the races. The downside was that when you wanted to work on your site, all the navigation was missing, because you were doing it on your regular computer without Apache, and your web browser thought those were just regular HTML comments. It was impossible to install Apache, of course, because you had a computer, not a server.

Sadly, that’s all gone now — paved over by homogenous timelines where anything that wasn’t made this week is old news and long forgotten. The web was supposed to make information eternal, but instead, so much of it became ephemeral. I miss when virtually everyone I knew had their own website. Having a Twitter and an Instagram as your entire online presence is a poor substitute.

So, let’s look at the Space Jam website.

Case study: Space Jam

Space Jam, if you’re not aware, is the greatest movie of all time. It documents Bugs Bunny’s extremely short-lived basketball career, playing alongside a live action Michael Jordan to save the planet from aliens for some reason. It was followed by a series of very successful and critically acclaimed RPG spinoffs, which describe the fallout of the Space Jam and are extremely canon.

And we are truly blessed, for 24 years after it came out, its website is STILL UP. We can explore the pinnacle of 1996 web design, right here, right now.

First, notice that every page of this site is a static page. Not only that, but it’s a static page ending in .htm rather than .html, because people on Windows versions before 95 were still beholden to 8.3 filenames. Not sure why that mattered in a URL, as if you were going to run Windows 3.11 on a Web server, but there you go.

The CSS for the splash page looks like this:

1
<body bgcolor="#000000" background="img/bg_stars.gif" text="#ff0000" link="#ff4c4c" vlink="#ff4c4c" alink="#ff4c4c">

Haha, just kidding! What the fuck is CSS? Space Jam predates it by a month. (I do see a single line in the page source, but I’m pretty sure that was added much later to style some legally obligatory policy links.)

Notice the extremely precise positioning of these navigation links. This feat was accomplished the same way everyone did everything in 1996: with tables.

In fact, tables have one functional advantage over CSS for layout, which was very important in those days, and not only because CSS didn’t exist yet. You see, you can ctrl-click to select a table cell and even drag around to select all of them, which shows you how the cells are arranged and functions as a super retro layout debugger. This was great because the first meaningful web debug tool, Firebug, wasn’t released until 2006 — a whole decade later!

Screenshot of the Space Jam website with the navigation table's cells selected, showing how the layout works

The markup for this table is overflowing with inexplicable blank lines, but with those removed, it looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
<table width=500 border=0>
<TR>
<TD colspan=5 align=right valign=top>
</td></tr>
<tr>
<td colspan=2 align=right valign=middle>
<br>
<br>
<br>
<a href="cmp/pressbox/pressboxframes.html"><img src="img/p-pressbox.gif" height=56 width=131 alt="Press Box Shuttle" border=0></a>
</td>
<td align=center valign=middle>
<a href="cmp/jamcentral/jamcentralframes.html"><img src="img/p-jamcentral.gif" height=67 width=55 alt="Jam Central" border=0></a>
</td>
<td align=center valign=top>
<a href="cmp/bball/bballframes.html"><img src="img/p-bball.gif" height=62 width=62 alt="Planet B-Ball" border=0></a>
</td>
<td align=center valign=bottom>
<br>
<br>
<a href="cmp/tunes/tunesframes.html"><img src="img/p-lunartunes.gif" height=77 width=95 alt="Lunar Tunes" border=0></a>
</td>
</tr>
<tr>
<td align=middle valign=top>
<br>
<br>
<a href="cmp/lineup/lineupframes.html"><img src="img/p-lineup.gif" height=52 width=63 alt="The Lineup" border=0></a>
</td>
<td colspan=3 rowspan=2 align=right valign=middle>
<img src="img/p-jamlogo.gif" height=165 width=272 alt="Space Jam" border=0>
</td>
<td align=right valign=bottom>
<a href="cmp/jump/jumpframes.html"><img src="img/p-jump.gif" height=52 width=58 alt="Jump Station" border=0></a>
</td>
</tr>
...
</table>

That’s the first two rows, including the logo. You get the idea. Everything is laid out with align and valign on table cells; rowspans and colspans are used frequently; and there are some <br>s thrown in for good measure, to adjust vertical positioning by one line-height at a time.

Other fantastic artifacts to be found on this page include this header, which contains Apache SSI syntax! This must’ve quietly broken when the site was moved over the years; it’s currently hosted on Amazon S3. You know, Amazon? The bookstore?

1
2
3
4
5
6
7
<table border=0 cellpadding=0 cellspacing=0 width=488 height=60>
<tr>
<td align="center"><!--#include virtual="html.ng/site=spacejam&type=movie&home=no&size=234&page.allowcompete=no"--></td>
<td align="center" width="20"></td>
<td align="center"><!--#include virtual="html.ng/site=spacejam&type=movie&home=no&size=234"--></td>
</tr>
</table>

Okay, let’s check out jam central. I’ve used my browser dev tools to reduce the viewport to 640×480 for the authentic experience (although I’d also have lost some vertical space to the title bar, taskbar, and five or six IE toolbars).

Note the frames: the logo in the top left leads back to the landing page, cleverly saving screen space on repeating all that navigation, and the top right is a fucking ad banner which has been blocked like seven different ways. All three parts are separate pages.

Screenshot of the Space Jam website's 'Jam Central'

Note also the utterly unreadable red text on a textured background, one of the truest hallmarks of 90s web design. “Why not put that block of text on an easier-to-read background?” you might ask. You imbecile. How would I possibly do that? Only the <body> has a background attribute! I could use a table, but tables only support solid background colors, and that would look so boring!

But wait, what is this new navigation widget? How are the links all misaligned like that? Is this yet another table? Well, no, although filling a table with chunks of a sliced-up image wasn’t uncommon. But this is an imagemap, a long-forgotten HTML feature. I’ll just show you the source:

1
2
3
4
5
6
7
8
<img src="img/m-central.jpg" height=301 width=438 border=0 alt="navigation map" usemap="#map"><br>

<map name="map">
<area shape="rect" coords="33,92,178,136" href="prodnotesframes.html" target="_top">
<area shape="rect" coords="244,111,416,152" href="photosframes.html" target="_top">
<area shape="rect" coords="104,138,229,181" href="filmmakersframes.html" target="_top">
<area shape="rect" coords="230,155,334,197" href="trailerframes.html" target="_top">
</map>

I assume this is more or less self-explanatory. The usemap attribute attaches an image map, which is defined as a bunch of clickable areas, beautifully encoded as inscrutable lists of coordinates or something.

And this stuff still works! This is in HTML! You could use it right now! Probably don’t though!

The thumbnail grid

Let’s look at one more random page here. I’d love to see some photos from the film. (Wait, photos? Did we not know what “screenshots” were yet?)

Screenshot of the Space Jam website's photos page

Another frameset, but arranged differently this time.

1
<body bgcolor="#7714bf" background="img/bg-jamcentral.gif" text="#ffffff" link="#edb2fc" vlink="#edb2fc" alink="#edb2fc">

They did an important thing here: since they specified a background image (which is opaque), they also specified a background color. Without it, if the background image failed to load, the page would be white text on the default white background, which would be unreadable.

(That’s still an important thing to keep in mind. I feel like modern web development tends to assume everything will load, or sees loading as some sort of inconvenience to be worked around, but not everyone is working on a wired connection in a San Francisco office twenty feet away from a backbone.)

But about the page itself. Thumbnail grids are a classic problem of web design, dating all the way back to… er… well, at least as far back as Space Jam. The main issue is that you want to put things next to each other, whereas HTML defaults to stacking everything in one big column. You could put all the thumbnails inline, in a single row of (wrapping) text, but that wouldn’t be much of a grid — and you usually want each one to have some sort of caption.

Space Jam’s approach was to use the only real tool anyone had in their toolbox at the time: a table. It’s structured like this:

1
2
3
4
5
<table cellpadding=10>
<tr><td align=center><a href="..."><img src="..."></a></td>...</tr>
<tr>...</tr>
<tr>...</tr>
<table>

A 3×3 grid of thumbnails, left to the browser to arrange. (The last image, on a row of its own, isn’t actually part of the table.) This can’t scale to fit your screen, but everyone’s screen was pretty tiny back then, so that was slightly less of a concern. They didn’t add captions here, but since every thumbnail is wrapped in a table cell, they easily could have.

This was the state of the art in thumbnail grids in 1996. We’ll be revisiting this little UI puzzle a few times; you can see live examples (and view source for sample markup) on a separate page.

But let’s take a moment to appreciate the size of the “full-size, full-color, internet-quality” movie screenshots on my current monitor.

Screenshot of one of the Space Jam website's full-size photos, fullscreened on my monitor

Hey, though, they’re less than 16 KB! That’ll only take nine seconds to download.

(I’m reminded of the problem of embedded video, which wasn’t solved until HTML5’s <video> tag some years later. Until then, you had to use a binary plugin, and all of them were terrible.)

(Oh, by the way: images within links, by default, have a link-colored border around them. Image links are usually self-evident, so this was largely annoying, and until CSS you had to disable them for every single image with <img border=0>.)

The regular early days

So that’s where we started, and it sucked. If you wanted any kind of consistency on more than a handful of pages, your options were very limited, and they were pretty much limited to a whole lot of copying and pasting. The Space Jam website opted to, for the most part, not bother at all — as did many others.

Then CSS came along, it was a fucking miracle. All that inline repetition went away. You want all your top-level headings to be a particular color? No problem:

1
2
3
H1 {
    color: #FF0000;
}

Bam! You’re done. No matter how many <h1>s you have in your document, every single one of them will be eye-searing red, and you never have to think about it again. Even better, you can put that snippet in its own file and have that questionable aesthetic choice applied to every page of your whole site with almost no effort! The same applied to your gorgeous tiling background image, the colors of your links, and the size of the font in your tables.

(Just remember to wrap the contents of your <style> tags in HTML comments, or old browsers without CSS support will display them as text.)

You weren’t limited to styling tags en masse, either. CSS introduced “classes” and “IDs” to target only specifically flagged elements. A selector like P.important would only affect <P CLASS="important">, and #header would only affect <H1 ID="header">. (The difference is that IDs are intended to be unique in a document, whereas classes can be used any number of times.) With these tools, you could effectively invent your own tags, giving you a customized version of HTML specific to your website!

This was a huge leap forward, but at the time, no one (probably?) was thinking of using CSS to actually arrange the page. When CSS 1 was made a recommendation in December ‘96, it barely addressed layout at all. All it did was divorce HTML’s existing abilities from the tags they were attached to. We had font colors and backgrounds because <FONT COLOR> and <BODY BACKGROUND> existed. The only feature that even remotely affected where things were positioned was the float property, the equivalent to <IMG ALIGN>, which pulled an image to the side and let text flow around it, like in a magazine article. Hardly whelming.

This wasn’t too surprising. HTML hadn’t had any real answers for layout besides tables, and the table properties were too complicated to generalize in CSS and too entangled with the tag structure, so there was nothing for CSS 1 to inherit. It merely reduced the repetition in what we were already doing with e.g. <FONT> tags — making Web design less tedious, less error-prone, less full of noise, and much more maintainable. A pretty good step forward, and everyone happily adopted it for that, but tables remained king for arranging your page.

That was okay, though; all your blog really needed was a header and a sidebar, which tables could do just fine, and it wasn’t like you were going to overhaul that basic structure very often. Copy/pasting a few lines of <TABLE BORDER=0> and <TD WIDTH=20%> wasn’t nearly as big a deal.

For some span of time — I want to say a couple years, but time passes more slowly when you’re a kid — this was the state of the Web. Tables for layout, CSS for… well, style. Colors, sizes, bold, underline. There was even this sick trick you could do with links where they’d only be underlined when the mouse was pointing at them. Tubular!

(Fun fact: HTML email is still basically trapped in this era.)

(And here’s about where I come in, at the ripe old age of 11, with no clue what I was doing and mostly learning from other 11-year-olds who also had no clue what they were doing. But that was fine; a huge chunk of the Web was 11-year-olds making their own websites, and it was beautiful. Why would you go to a business website when you can take a peek into the very specific hobbies of someone on the other side of the planet?)

The dark times

A year and a half later, in mid ‘98, we were gifted CSS 2. (I love the background on this page, by the way.) This was a modest upgrade that addressed a few deficiencies in various areas, but most interesting was the addition of a couple positioning primitives: the position property, which let you place elements at precise coordinates, and the inline-block display mode, which let you stick an element in a line of text like you could do with images.

Such tantalizing fruit, just out of reach! Using position seemed nice, but pixel-perfect positioning was at serious odds with the fluid design of HTML, and it was difficult to make much of anything that didn’t fall apart on other screen sizes or have other serious drawbacks. This humble inline-block thing seemed interesting enough; after all, it solved the core problem of HTML layout, which is putting things next to each other. But at least for the moment, no browser implemented it, and it was largely ignored.

I can’t say for sure if it was the introduction of positioning or some other factor, but something around this time inspired folks to try doing layout in CSS. Ideally, you would completely divorce the structure of your page from its appearance. A website even came along to take this principle to the extreme — CSS Zen Garden is still around, and showcases the same HTML being radically transformed into completely different designs by applying different stylesheets.

Trouble was, early CSS support was buggy as hell. In retrospect, I suspect browser vendors merely plucked the behavior off of HTML tags and called it a day. I’m delighted to say that RichInStyle still has an extensive list of early browser CSS bugs up; here are some of my favorites:

  • IE 3 would ignore all but the last <style> tag in a document.

  • IE 3 ignored pseudo-classes, so a:hover would be treated as a.

  • IE 3 and IE 4 treated auto margins as zero. Actually, I think this one might’ve persisted all the way to IE 6. But that was okay, because IE 6 also incorrectly applied text-align: center to block elements.

  • If you set a background image to an absolute URL, IE 3 would try to open the image in a local program, as though you’d downloaded it.

  • Netscape 4 understood an ID selector like #id, but ignored h1#id as invalid.

  • Netscape 4 didn’t inherit properties — including font and text color! — into table cells.

  • Netscape 4 applied properties on <li> to the list marker, rather than the contents.

  • If the same element has both float and clear (not unreasonable), Netscape 4 for Mac crashes.

This is what we had to work with. And folks wanted to use CSS to lay out an entire page? Ha.

Yet the idea grew in popularity. It even became a sort of elitist rallying cry, a best practice used to beat other folks over the head. Tables for layout are just plain bad, you’d hear! They confuse screenreaders, they’re semantically incorrect, they interact poorly with CSS positioning! All of which is true, but it was a much tougher pill to swallow when the alternative was—

Well, we’ll get to that in a moment. First, some background on the Web landscape circa 2000.

The end of the browser wars and subsequent stagnation

The short version is: this company Netscape had been selling its Navigator browser (to businesses; it was free for personal use), and then Microsoft entered the market with its completely free Internet Explorer browser, and then Microsoft had the audacity to bundle IE with Windows. Can you imagine? An operating system that comes with a browser? This was a whole big thing, Microsoft was sued over it, and they lost, and the consequence was basically nothing.

But it wouldn’t have mattered either way, because they’d still done it, and it had worked. IE pretty much annihilated Netscape’s market share. Both browsers were buggy as hell, and differently buggy as hell, so a site built exclusively against one was likely to be a big mess when viewed in the other — this meant that when Netscape’s market share dropped, web designers paid less and less attention to it, and less of the Web worked in it, and its market share dropped further.

Sucks for you if you don’t use Windows, I guess. Which is funny, because there was an IE for Mac 5.5, and it was generally less buggy than IE 6. (Incidentally, Bill Gates wasn’t so much a brilliant nerd as an aggressive and ruthless businessman who made his fortune by deliberately striving to annihilate any competition standing in his way and making computing worse overall as a result, just saying.)

By the time Windows XP shipped in mid 2001, with Internet Explorer 6 built in, Netscape had gone from a juggernaut to a tiny niche player.

And then, having completely and utterly dominated, Microsoft stopped. Internet Explorer had seen a release every year or so since its inception, but IE 6 was the last release for more than five years. It was still buggy, but that was less noticeable when there was no competition, and it was good enough. Windows XP, likewise, was good enough to take over the desktop, and there wouldn’t be another Windows for just as long.

The W3C, the group who write the standards (not to be confused with W3Schools, who are shady SEO leeches), also stopped. HTML had seen several revisions throughout the mid 90s, and then froze as HTML 4. CSS had gotten an update in only a year and a half, and then no more; the minor update CSS 2.1 wouldn’t hit Candidate Recommendation status until early 2004, and took another seven years to be finalized.

With IE 6’s dominance, it was as if the entire Web was frozen in time. Standards didn’t matter, because there was effectively only one browser, and whatever it did became the de facto standard. As the Web grew in popularity, IE’s stranglehold also made it difficult to use any platform other than Windows, since IE was Windows-only and it was a coin flip whether a website would actually work with any other browser.

(One begins to suspect that monopolies are bad. There oughta be a law!)

In the meantime, Netscape had put themselves in an even worse position by deciding to do a massive rewrite of their browser engine, culimating in the vastly more standards-compliant Netscape 6 — at the cost of several years away from the market while IE was kicking their ass. It never broke 10% market share, while IE’s would peak at 96%. On the other hand, the new engine was open sourced as the Mozilla Application Suite, which would be important in a few years.

Before we get to that, some other things were also happening.

Quirks mode

All early CSS implementations were riddled with bugs, but one in particular is perhaps the most infamous CSS bug of all time: the box model bug.

You see, a box (the rectangular space taken up by an element) has several measurements: its own width and height, then surrounding whitespace called padding, then an optional border, then a margin separating it from neighboring boxes. CSS specifies that these properties are all additive. A box with these styles:

1
2
3
    width: 100px;
    padding: 10px;
    border: 2px solid black;

…would thus be 124 pixels wide, from border to border.

IE 4 and Netscape 4, on the other hand, took a different approach: they treated width and height as measuring from border to border, and they subtracted the border and padding to get the width of the element itself. The same box in those browsers would be 100 pixels wide from border to border, with 76 pixels remaining for the content.

This conflict with the spec was not ideal, and IE 6 set out to fix it. Unfortunately, simply making the change would mean completely breaking the design of a whole lot of websites that had previously worked in both IE and Netscape.

So the IE team came up with a very strange compromise: they declared the old behavior (along with several other major bugs) as “quirks mode” and made it the default. The new “strict mode” or “standards mode” had to be opted into, by placing a “doctype” at the beginning of your document, before the <html> tag. It would look something like this:

1
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">

Everyone had to paste this damn mess of a line at the top of every single HTML document for years. (HTML5 would later simplify it to <!DOCTYPE html>.) In retrospect, it’s a really strange way to opt into correct CSS behavior; doctypes had been part of the HTML spec since way back when it was an RFC. I’m guessing the idea was that, since nobody bothered actually including one, it was a convenient way to allow opting in without requiring proprietary extensions just to avoid behavior that had been wrong in the first place. Good for the IE team!

The funny thing is, quirks mode still exists and is still the default in all browsers, twenty years later! The exact quirks have varied over time, and in particular neither Chrome nor Firefox use the IE box model even in quirks mode, but there are still quite a few other emulated bugs.

Modern browsers also have “almost standards” mode, which emulates only a single quirk, perhaps the second most infamous one: if a table cell contains only a single image, the space under the baseline is removed. Under normal CSS rules, the image is sitting within a line of (otherwise empty) text, which requires some space reserved underneath for descenders — the tails on letters like y. Early browsers didn’t handle this correctly, and some otherwise strict-mode websites from circa 2000 rely on it — e.g., by cutting up a large image and arranging the chunks in table cells, expecting them to display flush against each other — hence the intermediate mode to keep them limping along.

But getting back to the past: while this was certainly a win for standards (and thus interop), it created a new problem. Since IE 6 dominated, and doctypes were optional, there was little compelling reason to bother with strict mode. Other browsers ended up emulating it, and the non-standard behavior became its own de facto standard. Web designers who cared about this sort of thing (and to our credit, there were a lot of us) made a rallying cry out of enabling strict mode, since it was the absolute barest minimum step towards ensuring compatibility with other browsers.

The rise and fall of XHTML

Meanwhile, the W3C had lost interest in HTML in favor of developing XHTML, an attempt to redesign HTML with the syntax of XML rather than SGML.

(What on Earth is SGML, you ask? I don’t know. Nobody knows. It’s the grammer HTML was built on, and that’s the only reason anyone has heard of it.)

To their credit, there were some good reasons to do this at the time. HTML was generally hand-written (as it still is now), and anything hand-written is likely to have the occasional bugs. Browsers weren’t in the habit of rejecting buggy HTML outright, so they had various error-correction techniques — and, as with everything else, different browsers handled errors differently. Slightly malformed HTML might appear to work fine in IE 6 (where “work fine” means “does what you hoped for”), but turn into a horrible mess in anything else.

The W3C’s solution was XML, because their solution to fucking everything in the early 2000s was XML. If you’re not aware, XML takes a much more explicit and aggressive approach to error handling — if your document contains a parse error, the entire document is invalid. That means if you bank on XHTML and make a single typo somewhere, nothing at all renders. Just an error.

This sucked. It sounds okay on the face of things, but consider: generic XML is usually assembled dynamically with libraries that treat a document as a tree you manipulate, then turn it all into text when you’re done. That’s great for the common use of XML as data serialization, where your data is already a tree and much of the XML structure is simple and repetitive and easy to squirrel away in functions.

HTML is not like that. An HTML document has little reliable repeating structure; even this blog post, constructed mostly from <p> tags, also contains surprise <em>s within body text and the occasional <h2> between paragraphs. That’s not fun to express as a tree. And this is a big deal, because server-side rendering was becoming popular around the same time, and generated HTML was — still is! — put together with templates that treat it as a text stream.

If HTML were only written as complete static documents, then XHTML might have worked out — you write a document, you see it in your browser, you know it works, no problem. But generating it dynamically and risking that particular edge cases might replace your entire site with an unintelligible browser error? That sucks.

It certainly didn’t help that we were just starting to hear about this newfangled Unicode thing around this time, and it was still not always clear how exactly to make that work, and one bad UTF-8 sequence is enough for an entire XML document to be considered malformed!

And so, after some dabbling, XHTML was largely forgotten. Its legacy lives on in two ways:

  • It got us all to stop using uppercase tag names! So long <BODY>, hello <body>. XML is case-sensitive, you see, and all the XHTML tags were defined in lowercase, so uppercase tags simply would not work. (Fun fact: to this day, JavaScript APIs report HTML tag names in uppercase.) The increased popularity of syntax highlighting probably also had something to do with this; we weren’t all still using Notepad as we had been in 1997.

  • A bunch of folks still think self-closing tags are necessary. You see, HTML has two kinds of tags: containers like <p>...</p> and markers like <br>. Since a <br> can’t possibly contain anything, there’s no such thing as </br>. XML, as a generic grammar, doesn’t have this distinction; every tag must be closed, but as a shortcut, you can write <br/> to mean <br></br>.

    XHTML has been dead for years, but for some reason, I still see folks write <br/> in regular HTML documents. Outside of XML, that slash doesn’t do anything; HTML5 has defined it for compatibility reasons, but it’s silently ignored. It’s even actively harmful, since it might lead you to believe that <script/> is an empty <script> tag — but in HTML, it definitely is not!

I do miss one thing about XHTML. You could combine it with XSLT, the XML templating meta-language, to do in-browser templating (i.e., slot page-specific contents into your overall site layout) with no scripting required. It’s the only way that’s ever been possible, and it was cool as all hell when it worked, but the drawbacks were too severe when it didn’t. Also, XSLT is totally fucking incomprehensible.

The beginning of CSS layout

Back to CSS!

You’re an aspiring web designer. For whatever reason, you want to try using this CSS thing to lay out your whole page, even though it was clearly intended just for colors and stuff. What do you do?

As I mentioned before, your core problem is putting things next to each other. Putting things on top of each other is a non-problem — that’s the normal behavior of HTML. The whole reason everyone uses tables is that you can slop stuff into table cells and have it laid out side-by-side, in columns.

Well, tables seem to be out. CSS 2 had added some element display modes that corresponded to the parts of a table, but to use them, you’d have to have the same three levels of nesting as real tables: the table itself, then a row, then a cell. That doesn’t seem like a huge step up, and anyway, IE won’t support them until the distant future.

There’s that position thing, but it seems to make things overlap more often than not. Hmm.

What does that leave?

Only one tool, really: float.

I said that float was intended for magazine-style “pull” images, which is true, but CSS had defined it fairly generically. In principle, it could be applied to any element. If you wanted a sidebar, you could tell it to float to the left and be 20% the width of the page, and you’d get something like this:

1
2
3
4
+---------+
| sidebar | Hello, and welcome to my website!
|         |
+---------+

Alas! Floating has the secondary behavior that text wraps around it. If your page text was ever longer than your sidebar, it would wrap around underneath the sidebar, and the illusion would shatter. But hey, no problem. CSS specified that floats don’t wrap around each other, so all you needed to do was float the body as well!

1
2
3
4
5
6
7
+---------+ +-----------------------------------+
| sidebar | | Hello, and welcome to my website! |
|         | |                                   |
+---------+ | Here's a longer paragraph to show |
            | that my galaxy brain CSS float    |
            | nonsense prevents text wrap.      |
            +-----------------------------------+

This approach worked, but its limitations were much more obvious than those of tables. If you added a footer, for example, then it would try to fit to the right of the body text — remember, all of that is “pull” floats, so as far as the browser is concerned, the “cursor” is still at the top. So now you need to use clear, which bumps an element down below all floats, to fix that. And if you made the sidebar 20% wide and the body 80% wide, then any margin between them would add to that 100%, making the page wider than the viewport, so now you have an ugly horizontal scrollbar, so you have to do some goofy math to fix that as well. If you have borders or backgrounds on either part, then it was a little conspicuous that they were different heights, so now you have to do some truly grotesque stuff to fix that. And the more conscientious authors noticed that screenreaders would read the entire sidebar before getting to the body text, which is a pretty rude thing to subject blind visitors to, so they came up with yet more elaborate setups to have a three-column layout with the middle column appearing first in the HTML.

The result was a design that looked nice and worked well and scaled correctly, but backed by a weird mess of CSS. None of what you were writing actually corresponded to what you wanted — these are major parts of your design, not one-off pull quotes! It was difficult to understand the relationship between the layout-related CSS and what appeared on the screen, and that would get much worse before it got better.

Thumbnail grid 2

Armed with a new toy, we can improve that thumbnail grid. The original table-based layout was, even if you don’t care about tag semantics, incredibly tedious. Now we can do better!

1
2
3
4
5
6
<ul class="thumbnail-grid">
    <li><img src="..."><br>caption</li>
    <li><img src="..."><br>caption</li>
    <li><img src="..."><br>caption</li>
    ...
</ul>

This is the dream of CSS: your HTML contains the page data in some sensible form, and then CSS describes how it actually looks.

Unfortunately, with float as the only tool available to us, the results are a bit rough. This new version does adapt better to various screen sizes, but it requires some hacks: the cells have to be a fixed height, centering the whole grid is fairly complicated, and the grid effect falls apart entirely with wider elements. It’s becoming clear that what we wanted is something more like a table, but with a flexible number of columns. This is just faking it.

You also need this weird “clearfix” thing, an incantation that would become infamous during this era. Remember that a float doesn’t move the “cursor” — a fake idea I’m using, but close enough. That means that this <ul>, which is full only of floated elements, has no height at all. It ends exactly where it begins, with all the floated thumbnails spilling out below it. Worse, because any subsequent elements don’t have any floated siblings, they’ll ignore the thumbnails entirely and render normally from just below the empty “grid” — producing an overlapping mess!

The solution is to add a dummy element at the end of the list which takes up no space, but has the CSS clear: both — bumping it down below all floats. That effectively pushes the bottom of the <ul> under all the individual thumbnails, so it fits snugly around them.

Browsers would later support the ::before and ::after generated content” pseudo-elements, which let us avoid the dummy element entirely. Stylesheets from the mid-00s were often littered with stuff like this:

1
2
3
4
5
.thumbnail-grid::after {
    content: '';
    display: block;
    clear: both;
}

Still, it was better than tables.

DHTML

As a quick aside into the world of JavaScript, the newfangled position property did give us the ability to do some layout things dynamically. I heartily oppose such heresy, not least because no one has ever actually done it right, but it was nice for some toys.

Thus began the era of “dynamic HTML” — i.e., HTML affected by JavaScript, a term that has fallen entirely out of favor because we can’t even make a fucking static blog without JavaScript any more. In the early days it was much more innocuous, with teenagers putting sparkles that trailed behind your mouse cursor or little analog clocks that ticked by in real time.

The most popular source of these things was Dynamic Drive, a site that miraculously still exists and probably has a bunch of toys not updated since the early 00s.

But if you don’t like digging, here’s an example: every year (except this year when I forgot oops), I like to add confetti and other nonsense to my blog on my birthday. I’m very lazy so I started this tradition by using this script I found somewhere, originally intended for snowflakes. It works by placing a bunch of images on the page, giving them position: absolute, and meticulously altering their coordinates over and over.

Contrast this with the version I wrote from scratch a couple years ago, which has only a tiny bit of JS to set up the images, then lets the browser animate them with CSS. It’s slightly less featureful, but lets the browser do all the work, possibly even with hardware acceleration. How far we’ve come.

Web 2.0

Dark times can’t last forever. A combination of factors dragged us towards the light.

One of the biggest was Firefox — or, if you were cool, originally Phoenix and then Firebird — which hit 1.0 in Nov ‘04 and went on to take a serious bite out of IE. That rewritten Netscape 6 browser core, the heart of the Mozilla Suite, had been extracted into a standalone browser. It was quick, it was simple, it was much more standard-compliant, and absolutely none of that mattered.

No, Firefox really got a foothold because it had tabs. IE 6 did not have tabs; if you wanted to open a second webpage, you opened another window. It fucking sucked, man. Firefox was a miracle.

Firefox wasn’t the first tabbed browser, of course; the full Mozilla Suite’s browser had them, and the obscure (but scrappy!) Opera had had them for ages. But it was Firefox that took off, for various reasons, not least of which was that it didn’t have a giant fucking ad bar at the top like Opera did.

Designers did push for Firefox on standards grounds, of course; it’s just that that angle primarily appealed to other designers, not so much to their parents. One of the most popular and spectacular demonstrations was the Acid2 test, intended to test a variety of features of then-modern Web standards. It had the advantage of producing a cute smiley face when rendered correctly, and a fucking nightmare hellscape in IE 6. Early Firefox wasn’t perfect, but it was certainly much closer, and you could see it make progress until it fully passed with the release of Firefox 3.

It also helped that Firefox had a faster JavaScript engine, even before JIT caught on. Much, much faster. Like, as I recall, IE 6 implemented getElementById by iterating over the entire document, even though IDs are unique. Glance at some old jQuery release announcements; they usually have some performance charts, and everything else absolutely dwarfs IE 6 through 8.

Oh, and there was that whole thing where IE 6 was a giant walking security hole, especially with its native support for arbitrary binary components that only needed a “yes” click on an arcane dialog to get full and unrestricted access to your system. Probably didn’t help its reputation.

Anyway, with something other than IE taking over serious market share, even the most ornery designers couldn’t just target IE 6 and call it a day any more. Now there was a reason to use strict mode, a reason to care about compatibility and standards — which Firefox was making a constant effort to follow better, while IE 6 remained stagnant.

(I’d argue that this effect opened the door for OS X to make some inroads, and also for the iPhone to exist at all. I’m not kidding! Think about it; if the iPhone browser hadn’t actually worked with anything because everyone was still targeting IE 6, it’d basically have been a more expensive Palm. Remember, at first Apple didn’t even want native apps; it bet on the Web.)

(Speaking of which, Safari was released in Jan ‘03, based on a fork of the KHTML engine used in KDE’s Konqueror browser. I think I was using KDE at the time, so this was very exciting, but no one else really cared about OS X and its 2% market share.)

Another major factor appeared on April Fools’ Day, 2004, when Google announced Gmail. Ha, ha! A funny joke. Webmail that isn’t terrible? That’s a good one, Google.

Oh. Oh, fuck. Oh they’re not kidding. How the fuck does this even work

The answer, as every web dev now knows, is XMLHttpRequest — named for the fact that nobody has ever once used it to request XML. Apparently it was invented by Microsoft for use with Exchange, then cloned early on by Mozilla, but I’m just reading this from Wikipedia and you can do that yourself.

The important thing is, it lets you make an HTTP request from JavaScript. You could now update only part a page with new data, completely in the background, without reloading. Nobody had heard of this thing before, so when Google dropped an entire email client based on it, it was like fucking magic.

Arguably the whole thing was a mistake and has led to a hell future where static pages load three paragraphs of text in the background using XHR for no goddamn reason, but that’s a different post.

Along similar lines, August 2006 saw the release of jQuery, a similar miracle. Not only did it paper over the differences between IE’s “JScript” APIs and the standard approaches taken by everyone else (which had been done before by other libraries), but it made it very easy to work with whole groups of elements at a time, something that had historically been a huge pain in the ass. Now you could fairly easily apply CSS all over the place from JavaScript! Which is a bad idea! But everything was so bad that we did it anyway!

Hold on, I hear you cry. These things are about JavaScript! Isn’t this a post about CSS?

You’re absolutely right! I mention the rise of JavaScript because I think it led directly to the modern state of CSS, thanks to an increase in one big factor:

Ambition

Firefox showed us that we could have browsers that actually, like, improve — every new improvement on Acid2 was exciting. Gmail showed us that the Web could do more than show plain text with snowflakes in front.

And folks started itching to get fancy.

The problem was, browsers hadn’t really gotten any better yet. Firefox was faster in some respects, and it adhered more closely to the CSS spec, but it didn’t fundamentally do anything that browsers weren’t supposed to be able to do already. Only the tooling had improved, and that mostly affected JavaScript. CSS was a static language, so you couldn’t write a library to make it better. Generating CSS with JavaScript was a possibility, but boy oh boy is that ever a bad idea.

Another problem was that CSS 2 was only really good at styling rectangles. That was fine in the 90s, when every OS had the aesthetic of rectangles containing more rectangles. But now we were in the days of Windows XP and OS X, where everything was shiny and glossy and made of curvy plastic. It was a little embarrassing to have rounded corners and neatly shaded swooshes in your file browser and nowhere on the Web.

Thus began a new reign of darkness.

The era of CSS hacks

Designers wanted a lot of things that CSS just could not offer.

  • Round corners were a big one. Square corners had fallen out of vogue, and now everyone wanted buttons with round corners, since they were The Future. (Native buttons also went out of vogue, for some reason.) Alas, CSS had no way to do this. Your options were:

    1. Make a fixed-size background image of a rounded rectangle and put it on a fixed-size button. Maybe drop the text altogether and just make the whole thing an image. Eugh.

    2. Make a generic background image and scale it to fit. More clever, but the corners might end up not round.

    3. Make the rounded rectangle, cut out the corner and edges, and put them in a 3×3 table with the button label in the middle. Even better, use JavaScript to do this on the fly.

    4. Fuck it, make your entire website one big Flash app lol

    Another problem was that IE 6 didn’t understand PNGs with 8-bit alpha; it could only correctly display PNGs with 1-bit alpha, i.e. every pixel is either fully opaque or fully transparent, like GIFs. You had to settle for jagged edges, bake a solid background color into the image, or apply various fixes that centered around this fucking garbage nonsense:

    1
    filter: progid:DXImageTransform.Microsoft.AlphaImageLoader(src='bite-my-ass.png');
    
  • Along similar lines: gradients and drop shadows! You can’t have fancy plastic buttons without those. But here you were basically stuck with making images again.

  • Translucency was a bit of a mess. Most browsers supported the CSS 3 opacity property since very early on… except IE, which needed another wacky Microsoft-specific filter thing. And if you wanted only the background translucent, you’d need a translucent PNG, which… well, you know.

  • Since the beginning, jQuery shipped with built-in animated effects like fadeIn, and they started popping up all over the place. It was kind of like the Web equivalent of how every Linux user in the mid-00s (and I include myself in this) used that fucking Compiz cube effect.

    Obviously you need JavaScript to trigger an element’s disappearance in most interesting cases, but using it to control the actual animation was a bit heavy-handed and put a strain on browsers. Tabbed browsing compounded this, since browsers were largely single-threaded, and for various reasons, every open page ran in the same thread.

  • Oh! Alternating background colors on table rows. This has since gone out of style, but I think that’s a shame, because man did it make tables easier to read. But CSS had no answer for this, so you had to either give every other row a class like <tr class="odd"> (hope the table’s generated with code!) or do some jQuery nonsense.

  • CSS 2 introduced the > child selector, so you could write stuff like ul.foo > li to style special lists without messing up nested lists, and IE 6! Didn’t! Fucking! Support! It!

All those are merely aesthetic concerns, though. If you were interested in layout, well, the rise of Firefox had made your life at once much easier and much harder.

Remember inline-block? Firefox 2 actually supported it! It was buggy and hidden behind a vendor prefix, but it more or less worked, which let designers start playing with it. And then Firefox 3 supported it more or less fully, which felt miraculous. Version 3 of our thumbnail grid is as simple as a width and inline-block:

1
2
3
4
5
6
.thumbnails li {
    display: inline-block;
    width: 250px;
    margin: 0.5em;
    vertical-align: top;
}

The general idea of inline-block is that the inside acts like a block, but the block itself is placed in regular flowing text, like an image. Each thumbnail is thus contained in a box, but the boxes all lie next to each other, and because of their equal widths, they flow into a grid. And since it’s functionally a line of text, you don’t have to work around any weird impact on the rest of the page like you had to do with floats.

Sure, this had some drawbacks. You couldn’t do anything with the leftover space, for example, so there was a risk of a big empty void on the right with pathological screen sizes. You still had the problem of breaking the grid with a wide cell. But at least it’s not floats.

One teeny problem: IE 6. It did technically support inline-block, but only on elements that were naturally inline — ones like <b> and <i>, not <li>. So, not ones you’d actually want (or think) to use inline-block on. Sigh.

Lucky for us, at some point an absolute genius discovered hasLayout, an internal optimization in IE that marks whether an element… uh… has… layout. Look, I don’t know. Basically it changes the rendering path for an element — making it differently buggy, like quirks mode on a per-element basis! The upshot is that the above works in IE 6 if you add a couple lines:

1
2
3
4
5
6
7
8
.thumbnails li {
    display: inline-block;
    width: 250px;
    margin: 0.5em;
    vertical-align: top;
    *zoom: 1;
    *display: inline;
}

The leading asterisks make the property invalid, so browsers should ignore the whole line… but for some reason I cannot begin to fathom, IE 6 ignores the asterisks and accepts the rest of the rule. (Almost any punctuation worked, including a hyphen or — my personal favorite — an underscore.) The zoom property is a Microsoft extension that scales stuff, with the side effect that it grants the mystical property of “layout” to the element as well. And display: inline should make each element spill its contents into one big line of text, but IE treats an inline element that has “layout” roughly like an inline-block.

And here we saw the true potential of CSS messes. Browser-specific rules, with deliberate bad syntax that one browser would ignore, to replicate an effect that still isn’t clearly described by what you’re writing. Entire tutorials written to explain how to accomplish something simple, like a grid, but have it actually work on most people’s browsers. You’d also see * html, html > /**/ body, and all kinds of other nonsense. Here’s a full list! And remember that “clearfix” hack from before? The full version, compatible with every browser, is a bit worse:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
.clearfix:after {
  visibility: hidden;
  display: block;
  font-size: 0;
  content: " ";
  clear: both;
  height: 0;
}
.clearfix { display: inline-block; }
/* start commented backslash hack \*/
* html .clearfix { height: 1%; }
.clearfix { display: block; }
/* close commented backslash hack */

Is it any wonder folks started groaning about CSS?

This was an era of blind copy/pasting in the frustrated hopes of making the damn thing work. Case in point: someone (I dug the original source up once but can’t find it now) had the bone-headed idea of always setting body { font-size: 62.5% } due to a combination of “relative units are good” and wanting to override the seemingly massive default browser font size of 16px (which, it turns out, is correct) and dealing with IE bugs. He walked it back a short time later, but the damage had been done, and now thousands of websites start off that way as a “best practice”. Which means if you want to change your browser’s default font size in either direction, you’re screwed — scale it down and a bunch of the Web becomes microscopic, scale it up and everything will still be much smaller than you’ve asked for, scale it up more to compensate and everything that actually respects your decision will be ginormous. At least we have better page zoom now, I guess.

Oh, and do remember: Stack Overflow didn’t exist yet. This stuff was passed around purely by word of mouth. If you were lucky, you knew about some of the websites about websites, like quirks mode and Eric Meyer’s website.

In fact, check out Meyer’s css/edge site for some wild examples of stuff folks were doing, even with just CSS 1, as far back as 2002. I still think complexspiral is pure genius, even though you could do it nowadays with opacity and just one image. The approach in raggedfloat wouldn’t get native support in CSS until a few years ago, with shape-outside! He also brought us CSS reset, eliminating differences between browsers’ default styles.

(I cannot understand how much of a CSS pioneer Eric Meyer is. When his young daughter Rebecca died six years ago, she was uniquely immortalized with her own CSS color name, rebeccapurple. That’s how highly the Web community thinks of him. Also I have to go cry a bit over that story now.)

The future arrives, gradually

Designers and developers were pushing the bounds of what browsers were capable of. Browsers were handling it all somewhat poorly. All the fixes and workarounds and libraries were arcane, brittle, error-prone, and/or heavy.

Clearly, browsers needed some new functionality. But just slopping something in wouldn’t help; Microsoft had done plenty of that, and it had mostly made a mess.

Several struggling attempts began. With the W3C’s head still squarely up its own ass — even explicitly rejecting proposed enhancements to HTML, in favor of snorting XML — some folks from (active) browser vendors Apple, Mozilla, and Opera decided to make their own clubhouse. WHATWG came into existed in June 2004, and they began work on HTML5. (It would end up defining error-handling very explicitly, which completely obviated the need for XHTML and eliminated a number of security concerns when working with arbitrary HTML. Also it gave us some new goodies, like native audio, video, and form controls for dates and colors and other stuff that had been clumsily handled by JavaScript-powered custom controls. And, um, still often are.)

Then there was CSS 3. I’m not sure when it started to exist. It emerged slowly, struggling, like a chick hatching from an egg and taking its damn sweet fucking time to actually get implemented anywhere.

I’m having to do a lot of educated guessing here, but I think it began with border-radius. Specifically, with -moz-border-radius. I don’t know when it was first introduced, but the Mozilla bug tracker has mentions of it as far back as 1999.

See, Firefox’s own UI is rendered with CSS. If Mozilla wanted to do something that couldn’t be done with CSS, they added a property of their own, prefixed with -moz- to indicate it was their own invention. And when there’s no real harm in doing so, they leave the property accessible to websites as well.

My guess, then, is that the push for CSS 3 really began when Firefox took off and designers discovered -moz-border-radius. Suddenly, built-in rounded corners were available! No more fucking around in Photoshop; you only needed to write a single line! Practically overnight, everything everywhere had its corners filed down.

And from there, things snowballed. Common problems were addressed one at a time by new CSS features, which were clustered together into a new CSS version: CSS 3. The big ones were solutions to the design problems mentioned before:

  • Rounded corners, provided by border-radius.
  • Gradients, provided by linear-gradient() and friends.
  • Multiple backgrounds, which weren’t exactly a pressing concern, but which turned out to make some other stuff easier.
  • Translucency, provided by opacity and colors with an alpha channel.
  • Box shadows.
  • Text shadows, which had been in CSS 2 but dropped in 2.1 and never implemented anyway.
  • Border images, so you could do even fancier things than mere rounded borders.
  • Transitions and animations, now doable with ease without needing jQuery (or any JS at all).
  • :nth-child(), which solved the alternating rows problem with pure CSS.
  • Transformations. Wait, what? This kinda leaked in from SVG, which browsers were also being expected to implement, and which is built heavily around transforms. The code was already there, so, hey, now we can rotate stuff with CSS! Couldn’t do that before. Cool.
  • Web fonts, which had been in CSS for some time but only ever implemented in IE and only with some goofy DRM-laden font format. Now we weren’t limited to the four bad fonts that ship with Windows and that no one else has!

These were pretty great! They didn’t solve any layout problems, but they did address aesthetic issues that designers had been clumsily working around by using loads of images and/or JavaScript. That meant less stuff to download and more text used instead of images, both of which were pretty good for the Web.

The grand irony is that all the stuff you could do with these features went out of style almost immediately, and now we’re back to flat rectangles again.

Browser prefixing hell

Alas! All was still not right with the world.

Several of these new gizmos were, I believe, initially developed by browser vendors and prefixed. Some later ones were designed by the CSS committee but implemented by browsers while the design was still in flux, and thus also prefixed.

So began prefix hell, which continues to this day.

Mozilla had -moz-border-radius, so when Safari implemented it, it was named -webkit-border-radius (“WebKit” being the name of Apple’s KHTML fork). Then the CSS 3 spec standardized it and called it just border-radius. That meant that if you wanted to use rounded borders, you actually needed to give three rules:

1
2
3
4
5
element {
    -moz-border-radius: 1em;
    -webkit-border-radius: 1em;
    border-radius: 1em;
}

The first two made the effect actually work in current browsers, and the last one was future-proofing: when browsers implemented the real rule and dropped the prefixed ones, it would take over.

You had to do this every fucking time, since CSS isn’t a programming language and has no macros or functions or the like. Sometimes Opera and IE would have their own implementations with -o- and -ms- prefixes, bringing the total to five copies. It got much worse with gradients; the syntax went through a number of major incompatible revisions, so you couldn’t even rely on copy/pasting and changing the property name!

And plenty of folks, well, fucked it up. I can’t blame them too much; I mean, this sucks. But enough pages used only the prefixed forms, and not the final form, that browsers had to keep supporting the prefixed form for longer than they would’ve liked to avoid breaking stuff. And if the prefixed form still works and it’s what you’re used to writing, then maybe you still won’t bother with the unprefixed one.

Worse, some people would only use the form that worked in their pet choice of browser. This got especially bad with the rise of mobile web browsers. The built-in browsers on iOS and Android are Safari (WebKit) and Chrome (originally WebKit, now a fork), so you only “needed” to use the -webkit- properties. Which made things difficult for Mozilla when it released Firefox for Android.

Hey, remember that whole debacle with IE 6? Here we are again! It was bad enough that Mozilla eventually decided to implement a number of -webkit- properties, which remain supported even in desktop Firefox to this day. The situation is goofy enough that Firefox now supports some effects only via these properties, like -webkit-text-stroke, which isn’t being standardized.

Even better, Chrome’s current forked engine is called Blink, so technically it shouldn’t be using -webkit- properties either. And yet, here we are. At least it’s not as bad as the user agent string mess.

Browser vendors have pretty much abandoned prefixing, now; instead they hide experimental features behind flags (so they’ll only work on the developer’s machine), and new features are theoretically designed to be smaller and easier to stabilize.

This mess was probably a huge motivating factor for the development of Sass and LESS, two languages that produce CSS. Or… two CSS preprocessors, maybe. They have very similar goals: both add variables, functions, and some form of macros to CSS, allowing you to eliminate a lot of the repetition and browser hacks and other nonsense from your stylesheets. Hell, this blog still uses SCSS, though its use has gradually decreased over time.

Flexbox

But then, like an angel descending from heaven… flexbox.

Flexbox has been around for a long time — allegedly it had partial support in Firefox 2, back in 2006! It went through several incompatible revisions and took ages to stabilize. Then IE took ages to implement it, and you don’t really want to rely on layout tools that only work for half your audience. It’s only relatively recently (2015? Later?) that flexbox has had sufficiently broad support to use safely. And I could swear I still run into folks whose current Safari doesn’t recognize it at all without prefixing, even though Safari supposedly dropped the prefixes five years ago…

Anyway, flexbox is a CSS implementation of a pretty common GUI layout tool: you have a parent with some children, and the parent has some amount of space available, and it gets divided automatically between the children. You know, it puts things next to each other.

The general idea is that the browser computes how much space the parent has available and the “initial size” of each child, figures out how much extra space there is, and distributes it according to the flexibleness of each child. Think of a toolbar: you might want each button to have a fixed size (a flex of 0), but want to add spacers that share any leftover space equally, so you’d give them a flex of 1.

Once that’s done, you have a number of quality-of-life options at your disposal, too: you can distribute the extra space between the children instead, you can tell the children to stretch to the same height or align them in various ways, and you can even have them wrap into multiple rows if they won’t all fit!

With this, we can take yet another crack at that thumbnail grid:

1
2
3
4
5
6
7
.thumbnail-grid {
    display: flex;
    flex-wrap: wrap;
}
.thumbnail-grid li {
    flex: 1 0 250px;
}

This is miraculous. I forgot all about inline-block overnight and mostly salivated over this until it was universally supported. It even expresses very clearly what I want.

…almost. It still has the problem that too-wide cells will break the grid, since it’s still a horizontal row wrapped onto several independent lines. It’s pretty damn cool, though, and solves a number of other layout problems. Surely this is good enough. Unless…?

I’d say mass adoption of flexbox marked the beginning of the modern era of CSS. But there was one lingering problem…

The slow, agonizing death of IE

IE 6 took a long, long, long time to go away. It didn’t drop below 10% market share (still a huge chunk) until early 2010 or so.

Firefox hit 1.0 at the end of 2004. IE 7 wasn’t released until two years later, it offered only modest improvements, it suffered from compatibility problems with stuff built for IE 6, and the IE 6 holdouts (many of whom were not Computer People) generally saw no reason to upgrade. Vista shipped with IE 7, but Vista was kind of a flop — I don’t believe it ever came close to overtaking XP, not in its entire lifetime.

Other factors included corporate IT policies, which often take the form of “never upgrade anything ever” — and often for good reason, as I heard endless tales of internal apps that only worked in IE 6 for all manner of horrifying reasons. Then there was the entirety of South Korea, which was legally required to use IE 6 because they’d enshrined in law some security requirements that could only be implemented with an IE 6 ActiveX control.

So if you maintained a website that was used — or worse, required — by people who worked for businesses or lived in other countries, you were pretty much stuck supporting IE 6. Folks making little personal tools and websites abandoned IE 6 compatibility early on and plastered their sites with increasingly obnoxious banners taunting anyone who dared show up using it… but if you were someone’s boss, why would you tell them it’s okay to drop 20% of your potential audience? Just work harder!

The tension grew over the years, as CSS became more capable and IE 6 remained an anchor. It still didn’t even understand PNG alpha without workarounds, and meanwhile we were starting to get more critical features like native video in HTML5. The workarounds grew messier, and the list of features you basically just couldn’t use grew longer. (I’d show you what my blog looks like in IE 6, but I don’t think it can even connect — the TLS stuff it supports is so ancient and broken that it’s been disabled on most servers!)

Shoutouts, by the way, to some folks on the YouTube team, who in July 2009 added a warning banner imploring IE 6 users to switch to anything else — without asking anyone for approval. “Within one month… over 10 percent of global IE6 traffic had dropped off.” Not all heroes wear capes.

I’d mark the beginning of the end as the day YouTube actually dropped IE 6 support — March 13, 2010, almost nine years after its release. I don’t know how much of a direct impact YouTube has on corporate users or the South Korean government, but a massive web company dropping an entire browser sends a pretty strong message.

There were other versions of IE, of course, and many of them were messy headaches in their own right. But each subsequent one became less of a pain, and nowadays you don’t even have to think too much about testing in IE (now Edge). Just in time for Microsoft to scrap their own rendering engine and turn their browser into a Chrome clone.

Now

CSS is pretty great now. You don’t need weird fucking hacks just to put things next to each other. Browser dev tools are built in, now, and are fucking amazing — Firefox has started specifically warning you when some CSS properties won’t take effect because of the values of others! Obscure implicit side effects like “stacking contexts” (whatever those are) can now be set explicitly, with properties like isolation: isolate.

In fact, let me just list everything that I can think of that you can do in CSS now. This isn’t a guide to all possible uses of styling, but if your CSS knowledge hasn’t been updated since 2008, I hope this whets your appetite. And this stuff is just CSS! So many things that used to be impossible or painful or require clumsy plugins are now natively supported — audio, video, custom drawing, 3D rendering… not to mention the vast ergonomic improvements to JavaScript.

Layout

A grid container can do pretty much anything tables can do, and more, including automatically determining how many columns will fit. It’s fucking amazing. More on that below.

A flexbox container lays out its children in a row or column, allowing each child to declare its “default” size and what proportion of leftover space it wants to consume. Flexboxes can wrap, rearrange children without changing source order, and align children in a number of ways.

Columns will pour text into, well, multiple columns.

The box-sizing property lets you opt into the IE box model on a per-element basis, for when you need an entire element to take up a fixed amount of space and need padding/borders to subtract from that.

display: contents dumps an element’s contents out into its parent, as if it weren’t there at all. display: flow-root is basically an automatic clearfix, only a decade too late.

width can now be set to min-content, max-content, or the fit-content() function for more flexible behavior.

white-space: pre-wrap preserves whitespace, but breaks lines where necessary to avoid overflow. Also useful is pre-line, which collapses sequences of spaces down to a single space, but preserves literal newlines.

text-overflow cuts off overflowing text with an ellipsis (or custom character) when it would overflow, rather than simply truncating it. Also specced is the ability to fade out the text, but this is as yet unimplemented.

shape-outside alters the shape used when wrapping text around a float. It can even use the alpha channel of an image as the shape.

resize gives an arbitrary element a resize handle (as long as it has overflow).

writing-mode sets the direction that text flows. If your design needs to work for multiple writing modes, a number of CSS properties that mention left/right/top/bottom have alternatives that describe directions in terms of the writing mode: inset-block and inset-inline for position, block-size and inline-size for width/height, border-block and border-inline for borders, and similar for padding and margins.

Aesthetics

Transitions smoothly interpolate a value whenever it changes, whether due to an effect like :hover or e.g. a class being added from JavaScript. Animations are similar, but play a predefined animation automatically. Both can use a number of different easing functions.

border-radius rounds off the corners of a box. The corners can all be different sizes, and can be circular or elliptical. The curve also applies to the border, background, and any box shadows.

Box shadows can be used for the obvious effect of casting a drop shadow. You can also use multiple shadows and inset shadows for a variety of clever effects.

text-shadow does what it says on the tin, though you can also stack several of them for a rough approximation of a text outline.

transform lets you apply an arbitrary matrix transformation to an element — that is, you can scale, rotate, skew, translate, and/or do perspective transform, all without affecting layout.

filter (distinct from the IE 6 one) offers a handful of specific visual filters you can apply to an element. Most of them affect color, but there’s also a blur() and a drop-shadow() (which, unlike box-shadow, applies to an element’s appearance rather than its containing box).

linear-gradient(), radial-gradient(), the new and less-supported conic-gradient(), and their repeating-* variants all produce gradient images and can be used anywhere in CSS that an image is expected, most commonly as a background-image.

scrollbar-color changes the scrollbar color, with the downside of reducing the scrollbar to a very simple thumb-and-track in current browsers.

background-size: cover and contain will scale a background image proportionally, either big enough to completely cover the element (even if cropped) or small enough to exactly fit inside it (even if it doesn’t cover the entire background).

object-fit is a similar idea but for non-background media, like <img>s. The related object-position is like background-position.

Multiple backgrounds are possible, which is especially useful with gradients — you can stack multiple gradients, other background images, and a solid color on the bottom.

text-decoration is fancier than it used to be; you can now set the color of the line and use several different kinds of lines, including dashed, dotted, and wavy.

CSS counters can be used to number arbitrary elements in an arbitrary way, exposing the counting ability of <ol> to any set of elements you want.

The ::marker pseudo-element allows you to style a list item’s marker box, or even replace it outright with a custom counter. Browser support is spotty, but improving. Similarly, the @counter-style at-rule implements an entirely new counter style (like 1 2 3, i ii iii, A B C, etc.) which you can then use anywhere, though only Firefox supports it so far.

image-set() provides a list of candidate images and lets the browser choose the most appropriate one based on the pixel density of the user’s screen.

@font-face defines a font that can be downloaded, though you can avoid figuring out how to use it correctly by using Google Fonts.

pointer-events: none makes an element ignore the mouse entirely; it can’t be hovered, and clicks will go straight through it to the element below.

image-rendering can force an image to be resized nearest-neighbor rather than interpolated, though browser support is still spotty and you may need to also include some vendor-specific properties.

clip-path crops an element to an arbitrary shape. There’s also mask for arbitrary alpha masking, but browser support is spotty and hoo boy is this one complicated.

Syntax and misc

@supports lets you explicitly write different CSS depending on what the browser supports, though it’s nowhere near as useful nowadays as it would’ve been in 2004.

A > B selects immediate children. A + B selects siblings. A ~ B selects immediate (element) siblings. Square brackets can do a bunch of stuff to select based on attributes; most obvious is input[type=checkbox], though you can also do interesting things with matching parts of <a href>.

There are a whole bunch of pseudo-classes now. Many of them are for form elements: :enabled and :disabled; :checked and :indeterminate (also apply to radio and <option>); :required and :optional; :read-write and :read-only; :in-range/:out-of-range and :valid/:invalid (for use with HTML5 client-side form validation); :focus and :focus-within; and :default (which selects the default form button and any pre-selected checkboxes, radio buttons, and <option>s).

For targeting specific elements within a set of siblings, we have: :first-child, :last-child, and :only-child; :first-of-type, :last-of-type, and :only-of-type (where “type” means tag name); and :nth-child(), :nth-last-child(), :nth-of-type(), and :nth-last-of-type() (to select every second, third, etc. element).

:not() inverts a selector. :empty selects elements with no children and no text. :target selects the element jumped to with a URL fragment (e.g. if the address bar shows index.html#foo, this selects the element whose ID is foo).

::before and ::after should have two colons now, to indicate that they create pseudo-elements rather than merely scoping the selector they’re attached to. ::selection customizes how selected text appears; ::placeholder customizes how placeholder text (in text fields) appears.

Media queries do just a whole bunch of stuff so your page can adapt based on how it’s being viewed. The prefers-color-scheme media query tells you if the user’s system is set to a light or dark theme, so you can adjust accordingly without having to ask.

You can write translucent colors as #rrggbbaa or #rgba, as well as using the rgba() and hsla() functions.

Angles can be described as fractions of a full circle with the turn unit. Of course, deg and rad (and grad) are also available.

CSS variables (officially, “custom properties”) let you specify arbitrary named values that can be used anywhere a value would appear. You can use this to reduce the amount of CSS fiddling needs doing in JavaScript (e.g., recolor a complex part of a page by setting a CSS variable instead of manually adjusting a number of properties), or have a generic component that reacts to variables set by an ancestor.

calc() computes an arbitrary expression and updates automatically (though it’s somewhat obviated by box-sizing).

The vw, vh, vmin, and vmax units let you specify lengths as a fraction of the viewport’s width or height, or whichever of the two is bigger/smaller.


Phew! I’m sure I’m forgetting plenty and folks will have even longer lists of interesting tidbits in the comments. Thanks for saving me some effort! Now I can stop browsing MDN and do this final fun part.

State of the art thumbnail grid

At long last, we arrive at the final and objectively correct way to construct a thumbnail grid: using CSS grid. You can tell this is the right thing to use because it has “grid” in the name. Modern CSS features are pretty great about letting you say the thing you want and having it happen, rather than trying to coax it into happening implicitly via voodoo.

And it is oh so simple:

1
2
3
4
.thumbnail-grid {
    display: grid;
    grid: auto-flow / repeat(auto-fit, minmax(250px, 1fr));
}

Done! That gives you a grid. You have myriad other twiddles to play with, just as with flexbox, but that’s the basic idea. You don’t even need to style the elements themselves; most of the layout work is done in the container.

The grid shorthand property looks a little intimidating, but only because it’s so flexible. It’s saying: fill the grid one row at a time, generating as many rows as necessary; make as many 250px columns as will fit, and share any leftover space between them equally.

CSS grids are also handy for laying out <dl>s, something that’s historically been a massive pain to make work — a <dl> contains any number of <dt>s followed by any number of <dd>s (including zero), and the only way to style this until grid was to float the <dt>s, which meant they had to have a fixed width. Now you can just tell the <dt>s to go in the first column and <dd>s to go in the second, and grid will take care of the rest.

And laying out your page? That whole sidebar thing? Check out how easy that is:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
body {
    display: grid;
    grid-template:
        "header         header          header"
        "left-sidebar   main-content    right-sidebar"
        "footer         footer          footer"
        / 1fr           6fr             1fr
    ;
}
body > header {
    grid-area: header;
}
#left-sidebar {
    grid-area: left-sidebar;
}
/* ... etc ... */

Done. Easy. It doesn’t matter what order the parts appear in the markup, either.

On the other hand

The web is still a little bit of a disaster. A lot of folks don’t even know that flexbox and grid are supported almost universally now; but given how long it took to get from early spec work to broad implementation, I can’t really blame them. I saw a brand new little site just yesterday that consisted mostly of a huge list of “thumbnails” of various widths, and it used floats! Not even inline-block! I don’t know how we managed to teach everyone about all the hacks required to make that work, but somehow haven’t gotten the word out about flexbox.

But far worse than that: I still regularly encounter sites that do their entire page layout with JavaScript. If you use uMatrix, your first experience is with a pile of text overlapping a pile of other text. Surely this is a step backwards? What are you possibly doing that your header and sidebar can only be laid out correctly by executing code? It’s not like the page loads with no CSS — nothing in plain HTML will overlap by default! You have to tell it to do that!

And then there’s the mobile web, which despite everyone’s good intentions, has kind of turned out to be a failure. The idea was that you could use CSS media queries to fit your normal site on a phone screen, but instead, most major sites have entirely separate mobile versions. Which means that either the mobile site is missing a bunch of important features and I’ll have to awkwardly navigate that on my phone anyway, or the desktop site is full of crap that nobody actually needs.

(Meanwhile, Google’s own Android versions of Docs/Sheets/etc. have, like, 5% of the features of the Web versions? Not sure what to make of that.)

Hmm. Strongly considering writing something that goes more into detail about improvements to CSS since the Firefox 3 era, similar to the one I wrote for JavaScript. But this post is long enough.

Some futures that never were

I don’t know what’s coming next in CSS, especially now that flexbox and grid have solved all our problems. I’m vaguely aware of some work being done on more extensive math support, and possibly some functions for altering colors like in Sass. There’s a painting API that lets you generate backgrounds on the fly with JavaScript using the canvas API, which is… quite something. Apparently it’s now in spec that you can use attr() (which evaluates to the value of an HTML attribute) as the value for any property, which seems cool and might even let you implement HTML tables entirely in CSS, but you could do the same thing with variables. I mean, um, custom properties. I’m more excited about :is(), which matches any of a list of selectors, and subgrid, which lets you add some nesting to a grid but keep grandchildren still aligned to it.

Much easier is to list some things that were the future, but fizzled out.

  • display: run-in has been part of CSS since version 2 (way back in ‘98), but it’s basically unsupported. The idea is that a “run-in” box is inserted, inline, into the next block, so this:

    1
    2
    3
    <h2 style="display: run-in;">Title</h2>
    <p>Paragraph</p>
    <p>Paragraph</p>
    

    displays like this:

    Title Paragraph

    Paragraph

    And, ah, hm, I’m starting to see why it’s unsupported. It used to exist in WebKit, but was apparently so unworkable as to be removed six years ago.

  • Alternate stylesheets” were popular in the early 00s, at least on a few of my friends’ websites. The idea was that you could list more than one stylesheet for your site (presumably for different themes), and the browser would give the user a list of them. Alas, that list was always squirrelled away in a menu with no obvious indication of when it was actually populated, so in the end, everyone who wanted multiple themes just implemented an in-page theme switcher themselves.

    This feature is still supported, but apparently Chrome never bothered implementing it, so it’s effectively dead.

  • More generally, the original CSS spec clearly expects users to be able to write their own CSS for a website — right in paragraph 2 it says

    …the reader may have a personal style sheet to adjust for human or technological handicaps.

    Hey, that sounds cool. But it never materialized as a browser feature. Firefox has userContent.css and some URL selectors for writing per-site rules, but that’s relatively obscure.

    Still, there’s clearly demand for the concept, as evidenced by the popularity of the Stylish extension — which does just this. (Too bad it was bought by some chucklefucks who started using it to suck up browser data to sell to advertisers. Use Stylus instead.)

  • A common problem (well, for me) is that of styling the label for a checkbox, depending on its state. Styling the checkbox itself is easy enough with the :checked pseudo-selector. But if you arrange a checkbox and its label in the obvious way:

    1
    <label><input type="checkbox"> Description of what this does</label>
    

    …then CSS has no way to target either the <label> element or the text node. jQuery’s (originally custom) selector engine offered a custom :has() pseudo-class, which could be used to express this:

    1
    2
    3
    4
    /* checkbox label turns bold when checked */
    label:has(input:checked) {
        font-weight: bold;
    }
    

    Early CSS 3 selector discussions seemingly wanted to avoid this, I guess for performance reasons? The somewhat novel alternative was to write out the entire selector, but be able to alter which part of it the rules affected with a “subject” indicator. At first this was a pseudo-class:

    1
    2
    3
    label:subject input:checked {
        font-weight: bold;
    }
    

    Then later, they introduced a ! prefix instead:

    1
    2
    3
    !label input:checked {
        font-weight: bold;
    }
    

    Thankfully, this was decided to be a bad idea, so the current specced way to do this is… :has()! Unfortunately, it’s only allowed when querying from JavaScript, not in a live stylesheet, and nothing implements it anyway. 20 years and I’m still waiting for a way to style checkbox labels.

  • <style scoped> was an attribute that would’ve made a <style> element’s CSS rules only apply to other elements within its immediate parent, meaning you could drop in arbitrary (possibly user-written) CSS without any risk of affecting the rest of the page. Alas, this was quietly dropped some time ago, with shadow DOM suggested as a wildly inappropriate replacement.

  • I seem to recall that when I first heard about Web components, they were templates you could use to reduce duplication in pure HTML? But I can’t find any trace of that concept now, and the current implementations require JavaScript to define them, so there’s nothing declarative linking a new tag to its implementation. Which makes them completely unusable for anything that doesn’t have a compelling reason to rely on JS. Alas.

  • <blink> and <marquee>. RIP. Though both can be easily replicated with CSS animations.

That’s it

You’re still here? It’s over. Go home.

And maybe push back against Blink monoculture and use Firefox, including on your phone, unless for some reason you use an iPhone, which forbids other browser engines, which is far worse than anything Microsoft ever did, but we just kinda accept it for some reason.

Advent calendar 2019

Post Syndicated from Eevee original https://eev.ee/release/2019/12/01/advent-calendar-2019/

Calendar of things I made during December, with little screenshots

🔗 Advent calendar, with links to individual projects

Happy new year!

For December, I had the absolutely ludicrous idea to do an advent calendar, whereupon I would make and release a thing every day until Christmas.

It didn’t go quite as planned! But some pretty good stuff still came out of it.


Day 1: I started out well enough with the Doom text generator (and accompanying release post), which does something simple that I’ve wanted for a long time but never seen anywhere: generate text using the Doom font. Most of the effort here was just in hunting down the fonts and figuring out how they worked; the rest was gluing them together with the canvas API. It could be improved further, but it’s pretty solid and useful as-is!

Day 2: I tried another thing I’d always wanted: making a crossword! (Solve interactively on squares.io!) I didn’t expect it to take all day, but it did, and even then I found a typo that I didn’t have time to fix, and I had to rush with the clues. All in all, an entertaining but way too difficult first attempt. I’d love to try doing this more, though.

Day 3: I’ve made a couple SVG visualizations before — most notably in my post on Perlin noise — and decided to take another crack at it. The result was a visualization of all six modern trig functions, showing the relationships between them in two different ways. I’m pretty happy with how this turned out, and delighted that I learned some relationships I didn’t know about before, either! I do wish I’d drawn some of the similar triangles to make the relationships more explicit, but I ran out of time — just orienting the text correctly took ages, especially since a lot of it needed different placement in all four quadrants. I vaguely intended to get around to doing a couple more of these, but it didn’t end up happening.

Days 4 and 7: I love the PICO-8‘s built-in tracker, which makes way more sense to me than any “real” tracker, and set out to replicate it for the web. The result is PICOtracker! Unfortunately, this one didn’t get fully finished (yet) — it can play back sounds and music from the hardcoded Under Construction cart, but doesn’t support editing yet. Most of my time went to figuring out the Web Audio API, figuring out what the knobs in the PICO-8 tracker actually do (and shoutout to picolove for acting as source code reference), and figuring out how to weld the two together. I definitely want to revisit this in the near future!

Day 5: I’d been recently streaming Eternal Doom III and was almost done, and I keep being really lazy about putting Doom streams on YouTube, so I finished up the game (which took far, far longer than I expected) and posted the whole thing as a playlist. It spans like 24 hours. Good if you, uh, just want some Doom noise to listen to in the background.

Day 6: I’d expected Eternal Doom to be a quick day so I could have a break, and it was not. So I took an explicit day off.

Days 8 and 13: I made flathack, a web roguelike with only one floor! The idea came from having played NetHack a great many times, and having seen the first floor much more than any other part of the dungeon — so why not make that the whole game? It needs a lot more work, but I’m happy to have finally published a roguelike, and I think it already serves its intended purpose at least a little bit: it’s a cute little timewaster that doesn’t keep killing you.

Days 9–12: I got food poisoning. It sucked. A lot.

Days 14–20: Fresh off of making flathack in only two days, I got a bit too big for my britches and decided to try writing an interactive fiction game. In one day. Spoilers: it took more than one day. But I think the result is pretty charming: Star Anise Chronicles: Escape from the Chamber of Despair, a game about being a cat and causing wanton destruction, and also the first Star Anise Chronicles game to actually be published. A good chunk of the time was spent just drawing illustrations for it, which weren’t strictly necessary, but they add a lot to the game and they did get me back in an art mood.

Day 21: I feel like I’ve been scared of color for a long time, and that’s no good, so I drew and colored something.

Day 22: I drew some weird porn, and colored it too! Porn is just a blast to draw, and it’d been a while. I’ll let you find the link on the calendar if you really want it.

Day 23: Did not exist, due to becoming nocturnal.

Day 24–28: I started a big reference of a bunch of my Flora characters way back in November 2018, but I tried to paint it when I didn’t know what I wanted in a painting style, and eventually I gave up. Flat colors are better for references anyway, so I tried again, and this time I finished! I’m really happy with how it came out — I feel like I’m finally starting to get the hang of art, maybe, just as I hit five years of trying. Again, it’s wildly NSFW, but the link is on the calendar.


All told, I didn’t quite end up with 25 distinct things, but I did make some interesting stuff — some of which I’d been thinking about for a long time — and I’ll call that a success.

I’d love to get flathack to the point that it’s worth playing repeatedly, make more crosswords, and finish PICOtracker — but those will have to wait, since my GAMES MADE QUICK??? FOUR jam is coming up in a few days!

And speaking of which, I need to put a bunch of this stuff on Itch!

Old CSS, new CSS

Post Syndicated from Eevee original https://eev.ee/blog/2019/09/07/old-css-new-css/

I first got into web design/development in the late 90s, and only as I type this sentence do I realize how long ago that was.

And boy, it was horrendous. I mean, being able to make stuff and put it online where other people could see it was pretty slick, but we did not have very much to work with.

I’ve been taking for granted that most folks doing web stuff still remember those days, or at least the decade that followed, but I think that assumption might be a wee bit out of date. A little while ago I encountered a tweet incredulous at the lack of border-radius back in the day. I still remember waiting with bated breath for it to be unprefixed!

But then, I suspect I also know a number of folks who only tried web design in the old days, and assume nothing about it has changed since.

I’m here to tell all of you to get off my lawn. Here’s a history of CSS and web design, as I remember it.


The very early days

In the beginning, there was no CSS. This was very bad.

My favorite artifact of this era is the book I learned HTML from, HTML: The Definitive Guide, published in several editions in the mid to late 90s. The book was indeed about HTML, with no mention of CSS at all. I don’t have it any more and can’t readily find screenshots online, but here’s a page from HTML & XHTML: The Definitive Guide, which seems to be a revision (I’ll get to XHTML later) with much the same style. Here, then, is the cutting-edge web design advice of 199X:

Screenshot of a plain website in IE, with plain black text on a white background with a simple image

Clearly delineate headers and footers with horizontal rules.

No, that’s not a border-top. That’s an <hr>. The page title is almost certainly centered with, well, <center>.

The page uses the default text color, background, and font. Partly because this is a guidebook introducing concepts one at a time; partly because the book was printed in black and white; and partly, I’m sure, because it reflected the reality that coloring anything was a huge pain in the ass.

Let’s say you wanted all your <h1>s to be red, across your entire site. You had to do this:

1
<H1><FONT COLOR=red>...</FONT></H1>

every single goddamn time. Hope you never decide to switch to blue!

Oh, and everyone wrote HTML tags in all caps. I don’t remember why we all thought that was a good idea. Maybe this was before syntax highlighting in text editors was very common (read: I was 12 and using Notepad), and uppercase tags were easier to distinguish from body text.

Keeping your site consistent was thus something of a nightmare. One solution was to simply not style anything, which a lot of folks did. This was nice, in some ways, since browsers let you change those defaults, so you could read the web how you wanted.

A clever alternate solution, which I remember showing up in a lot of Geocities sites, was to simply give every page a completely different visual style. Fuck it, right? Just do whatever you want on each new page.

That trend was quite possibly the height of web design.

Damn, I miss those days. There were no big walled gardens, no Twitter or Facebook. If you had anything to say to anyone, you had to put together your own website. It was amazing. No one knew what they were doing; I’d wager that the vast majority of web designers at the time were clueless hobbyist tweens (like me) all copying from other clueless hobbyist tweens. Half the web was fan portals about Animorphs, with inexplicable splash pages warning you that their site worked best if you had a 640×480 screen. (Anyone else should, I don’t know, get a new monitor?) Everyone who was cool and in the know used Internet Explorer 3, the most advanced browser, but some losers still used Netscape Navigator so you had to put a “Best in IE” animated GIF on your splash page too.

This was also the era of “web-safe colors” — a palette of 216 colors, where every channel was one of 00, 33, 66, 99, cc, or ff — which existed because some people still had 256-color monitors! The things we take for granted now, like 24-bit color.

In fact, a lot of stuff we take for granted now was still a strange and untamed problem space. You want to have the same navigation on every page on your website? Okay, no problem: copy/paste it onto each page. When you update it, be sure to update every page — but most likely you’ll forget some, and your whole site will become an archaeological dig into itself, with strata of increasingly bitrotted pages.

Much easier was to use frames, meaning the browser window is split into a grid and a different page loads in each section… but then people would get confused if they landed on an individual page without the frames, as was common when coming from a search engine like AltaVista. (I can’t believe I’m explaining frames, but no one has used them since like 2001. You know iframes? The “i” is for inline, to distinguish them from regular frames, which take up the entire viewport.)

PHP wasn’t even called that yet, and nobody had heard of it. This weird “Perl” and “CGI” thing was really strange and hard to understand, and it didn’t work on your own computer, and the errors were hard to find and diagnose, and anyway Geocities didn’t support it. If you were really lucky and smart, your web host used Apache, and you could use its “server side include” syntax to do something like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
<BODY>
    <TABLE WIDTH=100% BORDER=0 CELLSPACING=8 CELLPADDING=0>
        <TR>
            <TD COLSPAN=2>
                <!--#include virtual="/header.html" --> 
            </TD>
        </TR>
        <TR>
            <TD WIDTH=20%>
                <!--#include virtual="/navigation.html" --> 
            </TD>
            <TD>
                (actual page content goes here)
            </TD>
        </TR>
    </TABLE>
</BODY>

Mwah. Beautiful. Apache would see the special comments, paste in the contents of the referenced files, and you’re off to the races. The downside was that when you wanted to work on your site, all the navigation was missing, because you were doing it on your regular computer without Apache, and your web browser thought those were just regular HTML comments. It was impossible to install Apache, of course, because you had a computer, not a server.

Sadly, that’s all gone now — paved over by homogenous timelines where anything that wasn’t made this week is old news and long forgotten. The web was supposed to make information eternal, but instead, so much of it became ephemeral. I miss when virtually everyone I knew had their own website. Having a Twitter and an Instagram as your entire online presence is a poor substitute.

So let’s look at the Space Jam website.

Case study: Space Jam

Space Jam, if you’re not aware, is the greatest movie of all time and focuses on Bugs Bunny’s brief basketball career, playing alongside a live action Michael Jordan. It was followed by a series of very successful and high-praised RPG spinoffs, which tell the story of the fallout of the Space Jam and are extremely canon.

And we are truly blessed, for 23 years after it came out, its website is STILL UP. We can explore the pinnacle of 1996 web design, right here, right now.

First, notice that every page of this site is a static page. Not only that, but it’s a static page ending in .htm rather than .html, because people on Windows versions before 95 were still beholden to 8.3 filenames. Not really sure why that mattered in a URL, but there you go.

The CSS for the splash page looks like this:

1
<body bgcolor="#000000" background="img/bg_stars.gif" text="#ff0000" link="#ff4c4c" vlink="#ff4c4c" alink="#ff4c4c">

Haha, just kidding! There’s no CSS at all. I see a single line in the page source, but I’m pretty sure that was added much later to style some policy links.

Next, notice the extremely precise positioning of these navigation links. This feat was accomplished the same way everyone did everything in 1996: with tables.

In fact, tables have one advantage over CSS for layout: you can ctrl-click to select a table cell and drag around to select all of them, which shows you how the cells are arranged and is kind of like a super retro layout debugger. This was great because the first meaningful web debug tool, Firebug, wasn’t released until 2006 — a whole ten years later!

Screenshot of the Space Jam website with the navigation table's cells selected, showing how the layout works

The markup for this table is overflowing with inexplicable blank lines, but with those removed, it looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
<table width=500 border=0>
<TR>
<TD colspan=5 align=right valign=top>
</td></tr>
<tr>
<td colspan=2 align=right valign=middle>
<br>
<br>
<br>
<a href="cmp/pressbox/pressboxframes.html"><img src="img/p-pressbox.gif" height=56 width=131 alt="Press Box Shuttle" border=0></a>
</td>
<td align=center valign=middle>
<a href="cmp/jamcentral/jamcentralframes.html"><img src="img/p-jamcentral.gif" height=67 width=55 alt="Jam Central" border=0></a>
</td>
<td align=center valign=top>
<a href="cmp/bball/bballframes.html"><img src="img/p-bball.gif" height=62 width=62 alt="Planet B-Ball" border=0></a>
</td>
<td align=center valign=bottom>
<br>
<br>
<a href="cmp/tunes/tunesframes.html"><img src="img/p-lunartunes.gif" height=77 width=95 alt="Lunar Tunes" border=0></a>
</td>
</tr>
<tr>
<td align=middle valign=top>
<br>
<br>
<a href="cmp/lineup/lineupframes.html"><img src="img/p-lineup.gif" height=52 width=63 alt="The Lineup" border=0></a>
</td>
<td colspan=3 rowspan=2 align=right valign=middle>
<img src="img/p-jamlogo.gif" height=165 width=272 alt="Space Jam" border=0>
</td>
<td align=right valign=bottom>
<a href="cmp/jump/jumpframes.html"><img src="img/p-jump.gif" height=52 width=58 alt="Jump Station" border=0></a>
</td>
</tr>
...
</table>

That’s the first two rows, including the logo. You get the idea. Everything is laid out with align and valign on table cells; rowspans and colspans are used frequently; and there are some <br>s thrown in for good measure, to adjust vertical positioning by one line-height at a time.

Other fantastic artifacts to be found on this page include this header, which contains Apache SSI syntax! This must’ve quietly broken when the site was moved over the years; it’s currently hosted on Amazon S3. You know, Amazon? The bookstore?

1
2
3
4
5
6
7
<table border=0 cellpadding=0 cellspacing=0 width=488 height=60>
<tr>
<td align="center"><!--#include virtual="html.ng/site=spacejam&type=movie&home=no&size=234&page.allowcompete=no"--></td>
<td align="center" width="20"></td>
<td align="center"><!--#include virtual="html.ng/site=spacejam&type=movie&home=no&size=234"--></td>
</tr>
</table>

Okay, let’s check out jam central. I’ve used my browser dev tools to reduce the viewport to 640×480 for the authentic experience (although I’d also have lost some vertical space to the title bar, taskbar, and five or six IE toolbars).

Note the frames: the logo in the top left leads back to the landing page, cleverly saving screen space on repeating all that navigation, and the top right is a fucking ad banner which has been blocked like seven different ways. All three parts are separate pages.

Screenshot of the Space Jam website's 'Jam Central'

Note also the utterly unreadable red text on a textured background, one of the truest hallmarks of 90s web design. “Why not put that block text on a solid background?” you might ask. You imbecile. How would I possibly do that? Only the <body> has a background attribute! I could use <table bgcolor>, but then I’d have to use a solid color, and that would look so boring!

But wait, what is this new navigation widget? How are the links all misaligned like that? Is this yet another table? Well, no, although filling a table with chunks of a sliced-up image wasn’t uncommon. But this is an imagemap, a long-forgotten HTML feature. I’ll just show you the source:

1
2
3
4
5
6
7
8
<img src="img/m-central.jpg" height=301 width=438 border=0 alt="navigation map" usemap="#map"><br>

<map name="map">
<area shape="rect" coords="33,92,178,136" href="prodnotesframes.html" target="_top">
<area shape="rect" coords="244,111,416,152" href="photosframes.html" target="_top">
<area shape="rect" coords="104,138,229,181" href="filmmakersframes.html" target="_top">
<area shape="rect" coords="230,155,334,197" href="trailerframes.html" target="_top">
</map>

I assume this is more or less self-explanatory. The usemap attribute attaches an image map, which is defined as a bunch of clickable areas, beautifully encoded as inscrutable lists of coordinates or something.

And this stuff still works! This is in HTML! You could use it right now!

Let’s look at one more random page here. I’d love to see some photos from the film. (Wait, photos? Did we not know what screenshots were yet?)

Screenshot of the Space Jam website's photos page

Another frameset, but arranged differently this time.

1
<body bgcolor="#7714bf" background="img/bg-jamcentral.gif" text="#ffffff" link="#edb2fc" vlink="#edb2fc" alink="#edb2fc">

They did an important thing here: since they specified a background image (which is opaque), they also specified a background color. Without it, if the background image failed to load, the page would be white text on the default white background, which would be unreadable.

This is still an important thing to keep in mind, by the way. I feel like modern web development tends to assume everything will load, or sees loading as some sort of inconvenience to be worked around.

Anyway, there’s not much to say here. The grid of thumbnails is, of course, done with a table. The thumbnails themselves are 72×43 pixels, and the “full-size, full-color, internet-quality” images are 360×216. Hey, though, they’re only like 16 KB! That’ll only take nine seconds to download.

The regular early days

So that’s where we started. This all sucked, obviously. If you wanted any kind of consistency on more than a handful of pages, your options were very limited.

And then CSS came along, it was a fucking miracle. You could put borders on stuff! You could set colors without having to copy-paste them everywhere! You could just write HTML and another file somewhere else would make it look like the rest of your website, which was important, because we still didn’t understand what “server-side scripting” was or how to use it to make anything more interesting than a hit counter! (I absolutely wrote a hit counter.)

A website even came along to take this principle to the extreme — CSS Zen Garden is still around, and showcases the same HTML being radically transformed into completely different designs by applying different stylesheets.

But we’re not quite there yet. First is CSS 1, made a recommendation in late ‘96.

I don’t know that this has been explicitly state, but CSS was clearly designed to divorce the existing layout and appearance capabilities of HTML away from the markup structure. That’s why even CSS 1 had the float property — it encapsulated the same functionality as the <img align> attribute. The white-space property exposed what <nobr> and <pre> did. That meant you could use a <p> to indicate a paragraph, and have it still mean “this is a paragraph” while styling it however you wanted.

Beyond that, though, there was very little to CSS 1. You could set the font, color, and background of basically anything (though color keywords weren’t even defined yet, and were left up to the implementation!); you could set margins, borders, padding, and width/height; and that was pretty much it. Even the position property didn’t exist yet.

So while CSS 1 was a tremendous help for trivial aesthetics like colors and fonts, it was useless for layout. Everyone picked it up as a really convenient way to keep the theme consistent, but tables remained king for actually laying out your page.

  • alt stylesheets!! when did those come along.
  • reader stylesheets, much much less useful nowadays

  • blink and marquee

css2 not until mid 98

  • browser wars, css included
  • nothing fucking worked, people stuck to tables for arranging things and css for light details
  • quirks mode! introduced by ie4? (list quirks, i love the img in a table cell one)

So, along came CSS, and it was a fucking miracle. Now you just needed to put that color in a single file:

1
2
3
h1 {
    color: navy;
}

And it would apply to every <h1> on every page you included that file in! This changes everything.

This is about where I come in and where my perspective really starts. But please bear in mind that I was like 11, with no idea what I was doing, talking to mostly other 11-year-olds. I’m sure the paid web devs putting together MSN or whatever had a slightly better grasp on things, but honestly — who the hell was going to business websites in 1998? Like, why would you even do that? The web was for random people to make their own little quirky beautiful things.

very, very long sigh …

Anyway, CSS basically killed all those <body> attributes and <font> tags and whatnot overnight. Budding web developers would get very haughty about how much better CSS was, and how you clearly had no idea what you were doing if you still used <font size=+1>. And so began best practice snobbery.

For some span of time — I want to say a couple years, but time passes weirdly when you’re a kid — this was the state of the web. Tables were still used for layout, but CSS was used for, well, style. Colors, sizes, bold, underline. There was even this sick trick you could do with links where they’d only be underlined when the mouse was pointing at them. Incredible!

But there were plenty of things you couldn’t do. Rather a lot, in fact. Here are some that I remember, and the workarounds we were stuck with.

  • You couldn’t, of course, put rounded corners on anything. You had to make four round-corner images yourself and put them in the corners of a 3×3 <table> (!). Or, if you knew the size of the element ahead of time, you could just make a single background image (!!). Of course, Internet Explorer 6 didn’t understand when an image in this newfangled “PNG” format had an 8-bit alpha channel, so you’d have to either use a GIF with jagged edges, or just include the page’s background color in the corner images so they could be opaque (!!!).

  • Speaking of opacity, that didn’t exist at all. The opacity property was new in CSS 3, and IE choked on PNGs, so if you wanted any sort of translucency, the solution was usually: don’t.

  • What the fuck is a “web font”? Your font options are, uh, whatever’s installed on your computer. I’m sure everyone else has the same fonts you do. Everyone’s using Windows XP, right?

    (There was once a time where people could configure their web browser to use fonts of their choosing, and it would actually matter, but those days seem to be long past. Alas, the stack of defaults means any website that doesn’t specify a font would be ugly for most people.)

  • No box shadows, no text shadows.

  • The +, ~, and > CSS combinators didn’t work in IE until 7, so they effectively didn’t exist.

  • XXX

I cannot stress enough that Stack Overflow did not yet exist. This stuff was picked up from various websites about websites, like quirks mode and Eric Meyer’s website.

(Eric Meyer is a CSS pioneer. When his young daughter Rebecca died five years ago, she was uniquely immortalized with a CSS color name, rebeccapurple. That’s how highly the web community thinks of him. Also I have to go cry a bit over that story now.)

XXX complexspiral?

The struggle begins

  • browser wars around the same time

Then, something happened. I don’t know how, or when, or why exactly. But people started to do website layout using CSS, too.

In hindsight, this was clearly absurd, and let me tell you why. CSS in its original form, which we’ll say for convenience is CSS 2.1, was designed like it was for articles — and nothing else. You could play with margins and font sizes and even colors, all you wanted.

But you could not put things next to each other.

Okay, that’s not entirely true, but it is mostly true.

CSS was, as I understand it, designed to extract out all the presentational stuff that people were already doing with HTML. So it inherited a lot of the HTML model, just abstracted away from specific tags. That meant there were really only two layout models. Inline layout was for stuff like <b> and <i>: variations on style within a line of wrapping text. Block layout was for stuff like <p> and <hr>: a vertical stack of elements that each stretched across the full width of the page (or whatever container).

So if you want to have a navigation sidebar on the left side of your website, what do you do?

You can’t use blocks, because those stack vertically. That leaves inline, but… that’s for flowing text, not putting large complex blocks next to each other. CSS had also introduced “absolute positioning” for sticking elements at a precise position on the page, but people had all kinds of different screen sizes and that approach was remarkably inflexible. What does that leave?

Only one option. And it’s only in hindsight that I can truly appreciate how ghastly this was.

You see, the HTML <img> element normally sits in the flow of text, as an inline element. But if you set its align attribute to left or right, the image would jump out of the flow of text and shift to one side of the document, with text flowing around it. And CSS had absorbed this functionality in a general way, as the float property.

CSS floats were only designed for one thing: flowing text around an image, like you might see in a magazine article. That’s it.

But they were also the only way to put two things next to each other. Because if you have two floats in the same place, then one will butt up against the other.

And so it begins.

The float hack era

Now, to be fair, this wasn’t a bad idea. Crafting layouts out of tables was hellish nonsense, and it suffered from the same problems as embedding colors in your HTML: if you ever wanted to make a slight adjustment, you’d have to repeat it on every single page. Surely, the ideal would be for the markup to describe the content and the CSS to describe how to arrange it on screen.

XXX how a float layout actually worked

XXX why it was brittle etc, but the biggest problem was:

The dreadnaught: Internet Explorer 6

released in 2001, right around when css layout concept was taking off. completely ate market share in XXX, over 95%. basically abandoned for the next five years

and we discovered some problems

and then firefox came along and was taking the world by storm so suddenly everyone had to make their carefully-hacked websites work in both ie6 and firefox, and that was proving to be a problem, because ie6 had a lot of fundamental and severe bugs

acid2!!

what were the biggest things in css3? mention those
– no web fonts, so people basically used whatever shipped with windows, which sucked if you weren’t on windows

using <!-- at the start of your stylesheet lmfao
the bad old days

i came in shortly before the long winter of ie6
browser wars
basically came down to just ie6 and a handful of people using like, konqueror on linux. (hi! ps that became webkit)
let me explain ie6 to you

ie6 box model, solved by strictmode? A LOT of stuff was “solved” by strict mode
– margin: auto treated like 0, but you could use text-align on the parent to center
“min-height” bug, no overflow it just makes stuff taller

no inline-block! i remember being very excited for firefox 4(?) for this reason, was a huge refactor by david baron

oh, the cross-browser hacks. oh my god.
parsing hacks: conditional comments, putting a squiggle before css prop name, ie6 can’t parse >
zoom
ie6 filters to fix stuff sometimes

xhtml??

shenanigans: complexspiral, etc

dhtml”, dynamicdrive (oreilly book from 2007) (dynamicdrive was 2000)

pushback against tables, which some folks didn’t really understand
it was a good idea but css was still designed for print, kinda, concerns, so didn’t have a ton of layout options
everything done with floats, the only way to put stuff side by side
“clearfix”, sob. could do it with ::after but i don’t think ie6 supported those

css reset

lot of stuff done in flash because it was easier

started making progress when firefox came out

the miraculous turning point came when ie6 was dead
but it was a slow, agonizing death, no real moment, kind of up to everyone individually
if you were doing web dev for a job, maybe even 2% ie6 traffic was enough to keep support for it
but wow what a nightmare that was

youtube dropped it March 13, 2010, which i think was the first serious big-name shun and imo marked the end

transitions? jquery. that’s it. an era where “modern” websites had everything sliding around with jquery effects, sort of like the linux 3d compiz cube thing

nowadays, css flexbox and grid, holy crap
rounded borders, sure
but also drop shadows, text shadows, transforms, transitions/animations, filters, svg filters

finally a css feature that lets me say what i want and have it happen, rather than try to coax it into happening implicitly”

fun facts:

(Fun fact: HTML email is still basically trapped in this era.)

grand irony: as soon as rounded glossy bubble buttons became easy to do with css, they went out of style! now we’re back to stuff we could’ve done pretty easily in 1996, except for the round avatars i guess

No More Forgetting to Input ERP Charges – Hello Automated ERP!

Post Syndicated from Grab Tech original https://engineering.grab.com/automated-erp-charges

ERP, standing for Electronic Road Pricing, is a system used to manage road congestion in Singapore. Drivers are charged when they pass through ERP gantries during peak hours. ERP rates vary for different roads and time periods based on the traffic conditions at the time. This encourages people to change their mode of transport, travel route or time of travel during peak hours. ERP is seen as an effective measure in addressing traffic conditions and ensuring drivers continue to have a smooth journey.

Did you know that Singapore has a total of 79 active ERP gantries? Did you also know that every ERP gantry changes its fare 10 times a day on average? For example, total ERP charges for a journey from Ang Mo Kio to Marina will cost $10 if you leave at 8:50am, but $4 if you leave at 9:00am on a working day!

Imagine how troublesome it would have been for Grab’s driver-partners who, on top of having to drive and check navigation, would also have had to remember each and every gantry they passed, calculating their total fare and then manually entering the charges to the total ride cost at the end of the ride.

In fact, based on our driver-partners’ feedback, missing out on ERP charges was listed as one of their top-most pain points. Not only did the drivers find the entire process troublesome, this also led to earnings loss as they would have had to bear the cost of the  ERP fares.

We’re glad to share that, as of 15th March 2019, we’ve successfully resolved this pain point for our driver-partners by introducing automated ERP fare calculation!

So, how did we achieve automating the ERP fare calculation for our drivers-partners? How did we manage to reduce the number of trips where drivers would forget to enter ERP fare to almost zero? Read on!

How we approached the Problem

The question we wanted to solve was – how do we create an impactful feature to make sure that driver -partners have one less thing to handle when they drive?

We started by looking at the problem at hand. ERP fares in Singapore are very dynamic; it changes on the basis of day and time.

Caption: Example of ERP fare changes on a normal weekday in Singapore
Caption: Example of ERP fare changes on a normal weekday in Singapore

 

We wanted to create a system which can identify the dynamic ERP fares at any given time and location, while simultaneously identifying when a driver-partner has passed through any of these gantries.

However, that wasn’t enough. We wanted this feature to be scalable to every country where Grab is in – like Indonesia, Thailand, Malaysia, Philippines, Vietnam. We started studying the ERP (or tolls – as it is known locally) system in other countries. We realized that every country has its own style of calculating toll. While in Singapore ERP charges for cars and taxis are the same, Malaysia applies different charges for cars and taxis. Similarly, Vietnam has different tolls for 4-seaters and 7-seaters. Indonesia and Thailand have couple gantries where you pay only at one of the gantries.Suppose A and B are couple gantries, if you passed through A, you won’t need to pay at B and vice versa. This is where our Ops team came to the rescue!

Boots on the Ground!

Collecting all the ERP or toll data for every country is no small feat, recalls Robinson Kudali, program manager for the project. “We had around 15 people travelling across the region for 2-3 weeks, working on collecting data from every possible source in every country.”

Getting the right geographical coordinates for every gantry is very important. We track driver GPS pings frequently, identify the nearest road to that GPS ping and check the presence of a gantry using its coordinates. The entire process requires you to be very accurate; incorrect gantry location can easily lead to us miscalculating the fare.

Bayu Yanuaragi, our regional mapops lead, explains – “To do this, the first step was to identify all toll gates for all expressways & highways in the country. The team used various mapping software to locate and plot all entry & exit gates using map sources, open data and more importantly government data as references. Each gate was manually plotted using satellite imagery and aligned with our road layers in order to extract the coordinates with a unique gantry ID.”

Location precision is vital in creating the dataset as it dictates whether a toll gate will be detected by the Grab app or not. Next step was to identify the toll charge from one gate to another. Accuracy of toll charge per segment directly reflects on the fare that the passenger pays after the trip.

Caption: ERP gantries visualisation on our map - The purple bars are the gantries that we drew on our map
Caption: ERP gantries visualisation on our map – The purple bars are the gantries that we drew on our map

 

Once the data compilation is done, team would then conduct fieldwork to verify its integrity. If data gaps are identified, modifications would be made accordingly.

Upon submission of the output, stack engineers would perform higher level quality check of the content in staging.

Lastly, we worked with a local team of driver-partners who volunteered to make sure the new system is fully operational and the prices are correct. Inconsistencies observed were reported by these driver-partners, and then corrected in our system.

Closing the loop

Creating a strong dataset did help us in predicting correct fares, but we needed something which allows us to handle the dynamic behavior of the changing toll status too. For example, Singapore government revises ERP fare every quarter, while there could also be ad-hoc changes like activating or deactivating of gantries on an on-going basis.

Garvee Garg, Product Manager for this feature explains: “Creating a system that solves the current problem isn’t sufficient. Your product should be robust enough to handle all future edge case scenarios too. Hence we thought of building a feedback mechanism with drivers.”

In case our ERP fare estimate isn’t correct or there are changes in ERPs on-ground, our driver-partners can provide feedback to us. These feedback directly flow to Customer Experience teamwho does the initial investigation, and from there to our Ops team. A dedicated person from Ops team checks the validity of the feedback, and recommends updates. It only takes 1 day on average to update the data from when we receive the feedback from the driver-partner.

However, validating the driver feedback was a time consuming process. We needed a tool which can ease the life of Ops team by helping them in de-bugging each and every case.

Hence the ERP Workflow tool came into the picture.

99% of the time, feedback from our driver-partners are about error cases. When feedback comes in, this tool would allow the Ops team to check the entire ride history of the driver and map driver’s ride trajectory with all the underlying ERP gantries at that particular point of time. The Ops team  would then be able to identify if ERP fare calculated by our system or as said by driver is right or wrong.

This is only the beginning

By creating a system that can automatically calculate and key in ERP fares for each trip, Grab is proud to say that our driver-partners can now drive with less hassle and focus more on the road which will bring the ride experience and safety for both the driver and the passengers to a new level!

The Automated ERP feature is currently live in Singapore and we are now testing it with our driver-partners in Indonesia and Thailand. Next up, we plan to pilot in the Philippines and Malaysia and soon to every country where Grab is in – so stay tuned for even more innovative ideas to enhance your experience on our super app!

To know more about what Grab has been doing to improve the convenience and experience for both our driver-partners and passengers, check out other stories on this blog!

Guiding you Door-to-Door via our Super App!

Post Syndicated from Grab Tech original https://engineering.grab.com/poi-entrances-venues-door-to-door

Remember landing at an airport or going to your favourite mall and the hassle of finding the pickup spot when you booked a cab? When there are about a million entrances, it can get particularly annoying trying to find the right pickup location!

Rolling out across South East Asia  is a brand new booking experience from Grab, designed  to make it easier for you to make a booking at large venues like airports, shopping centers, and tourist destinations! With the new booking flow, it will not only be easier to select one of the pre-designated Grab pickup points, you can also find text and image directions to help you navigate your way through the venue for a smoother rendezvous with your driver!

Inspiration behind the work

Finding your pick-up point closest to you, let alone predicting it, is incredibly challenging, especially when you are inside huge buildings or in crowded areas. Neeraj Mishra, Product Owner for Places at Grab explains: “We rely on GPS-data to understand user’s location which can be tricky when you are indoors or surrounded by skyscrapers. Since the satellite signal has to go through layers of concrete and steel, it becomes weak which adds to the inaccuracy. Furthermore, ensuring that passengers and drivers have the same pick-up point in mind can be tricky, especially with venues that have multiple entrances. ”  

Marina One POI

Grab’s data analysis revealed that “rendezvous distance” (walking distance between the selected pick-up point and where the car is waiting) is more than twice the Grab average when the booking is made from large venues such as airports.

To solve this issue, Grab launched “Entrances” (the green dots on the map) last year, which lists the various pick-up points available at a particular building, and shows them on the map, allowing users to easily choose the one closest to them, and ensuring their drivers know exactly where they want to be picked up from. Since then, Grab has created more than 120,000 such entrances, and we are delighted to inform you that average of rendezvous distances across all  countries have been steadily going down!

Decreasing rendezvous distance across region

One problem remained

But there was still one common pain-point to be solved. Just because a passenger has selected the pick-up point closest to them, doesn’t mean it’s easy for them to find it. This is particularly challenging at very large venues like airports and shopping centres, and especially difficult if the passenger is unfamiliar with the venue, for example – a tourist landing at Jakarta Airport for the very first time. To deliver an even smoother booking and pick-up experience, Grab has rolled out a new feature called Venues – the first in the region – that will give passengers in-app photo and text directions to the pick-up point closest to them.

Let’s break it down! How does it work?

Whether you are a local or a foreigner on holiday or business trip, fret not if you are not too familiar with the place that you are in!

Let’s imagine that you are now at Singapore Changi Airport: your new booking experience will look something like this!

Step 1: Fire the Grab app and click on Transport. You will see a welcome screen showing you where you are!

Welcome to Changi Airport

Step 2: On booking screen, you will see a new pickup menu with a list of available pickup points. Confirm the pickup point you want and make the booking!

Booking screen at Changi Airport

Step 3: Once you’ve been allocated a driver, tap on the bubble to get directions to your pick-up point!

Driver allocated at Changi Airport

Step 4: Follow the landmarks and walking instructions and you’ve arrived at your pick-up point!

Directions to pick-up point at Changi Airport

Curious about how we got this done?

Data-Driven Decisions

Based on a thorough data analysis of historical bookings, Grab identified key venues across our markets in Southeast Asia. Then we dispatched our Operations team to the ground, to identify all pick up points and perform detailed on-ground survey of the venue.

Operations Team’s Leg Work

Nagur Hassan, Operations Manager at Grab, explains the process: “For the venue survey process, we send a team equipped with the tools required to capture the details, like cameras, wifi and bluetooth scanners etc. Once inside the venue, the team identifies strategic landmarks and clear direction signs that are related to drop-off and pick-up points. Team also captures turn-by-turn walking directions to make it easier for Grab users to navigate – For instance, walk towards Starbucks and take a left near H&M store. All the photos and documentations taken on the sites are then brought back to the office for further processing.”

Quality Assurance

Once the data is collected, our in-house team checks the quality of the images and data. We also mask people’s faces and number plates of the vehicles to hide any identity-related information. As of today, we have collected 3400+ images for 1900+ pick up points belonging to 600 key venues! This effort took more than 3000 man-hours in total! And we aim to cover more than 10,000 such venues across the region in the next few months.

This is only the beginning

We’re constantly striving to improve the location accuracy of our passengers by using advanced Machine Learning and constant feedback mechanism. We understand GPS may not always be the most accurate determination of your current location, especially in crowded areas and skyscraper districts. This is just the beginning and we’re planning to launch some very innovative features in the coming months! So stay tuned for more!

Cheezball Rising: Collision detection, part 1

Post Syndicated from Eevee original https://eev.ee/blog/2018/11/28/cheezball-rising-collision-detection-part-1/

This is a series about Star Anise Chronicles: Cheezball Rising, an expansive adventure game about my cat for the Game Boy Color. Follow along as I struggle to make something with this bleeding-edge console!

GitHub has intermittent prebuilt ROMs, or you can get them a week early on Patreon if you pledge $4. More details in the README!


In this issue, I bash my head against a rock. Sorry, I mean I bash Star Anise against a rock. It’s about collision detection.

Previously: I draw some text to the screen.
Next: more collision detection, and fixed-point arithmetic.

Recap

Last time I avoided doing collision detection by writing a little dialogue system instead. It was cute, and definitely something that needed doing, but something much more crucial still looms.

Animation of the text box sliding up and scrolling out the text

I’ve put it off as long as I can. If I want to get anywhere with actual gameplay, I’m going to need some collision detection.

Background and upfront decisions

Collision detection is hard. It’s a lot of math that happens a few pixels at a time. Small mistakes can have dramatic consequences, yet be obscure enough that you don’t even notice them. Even using an off-the-shelf physics engine often requires dealing with a mountain of subtle quirks. And did I mention I have to do it on a Game Boy?

Someday I’ll write an article about everything I’ve picked up about collision detection, but I haven’t yet, so you get the quick version. The problem is that an object is moving around, and it should be unable to move into solid objects. There are two basic schools of thought about the solution.


Discrete collision observes that an object moves in steps — a little chunk of movement every frame — and simply teleports the object to its new location, then checks whether it now overlaps anything.

Illustration of an object attempting to move into a wall

(Note that all of these diagrams show very exaggerated motion. In most games, objects are slow and frames are short, so nothing moves more than a pixel or two at a time. That’s another reason collision detection is hard: the steps are so small that it can be difficult to see what’s actually going on.)

If it does overlap, you might might try to push it out of whatever it’s overlapping, or you might cancel the movement entirely and simply not move the object that frame.

Both approaches have drawbacks. Pushing an object out of an obstacle isn’t too difficult a problem, but it’s possible that the object will be pushed out into another obstacle, and now you have a complicated problem. (At this point, though, you could just give up and fall back to cancelling the movement.)

But cancelling the movement means that an object might get “stuck” a pixel or two away from a wall and never be able to butt up against it. The faster the object is trying to move, the bigger the risk that this might happen.

That said, this is exactly how the original Doom engine handles collision, and it seems to work well enough there. On the other hand, Doom is first-person so you can’t easily tell if you’re butting right up against a wall; a pixel gap is far more obvious in a game like this. On the other other hand, Doom also has bugs where a fast monster can open a locked door from its other side, because the initial teleport briefly moves the monster far enough into the door that it’s touching the other (unlocked) side.

Sorry. I have very conflicting feelings about this thicket of drawbacks and possible workarounds.

Either way, discrete collision has one other big drawback: tunnelling. Since the movement is done by teleporting, a very fast object might teleport right past a thin barrier. Only the new position is checked for collisions, so the barrier is never noticed. (This is how you travel to parallel universes in Mario 64 — by building up enough speed that Mario teleports through walls without ever momentarily overlapping them.)

Illustration of an object passing through a wall or erroneously pushing into one

There are some other potential gotchas, though they’re rare enough that I’ve never seen anyone mention them. One that stands out to me is that you don’t know the order that an object collided with obstacles, which might make a difference if the obstacles have special behavior when collided with and the order of that behavior matters.


Continuous collision detection observes that game physics are trying to simulate continuous motion, like happens in the real world, and tries to apply that to movement as well. Instead of teleporting, objects slide until they hit something. Tunnelling is thus impossible, and there’s no need to handle collisions since they’re prevented in the first place.

Illustration of an object sliding towards a wall and stopping when it touches

This has some clear advantages, in that it eliminates all the pitfalls of discrete collision! It even functions as a superset — if you want some object to act discretely, you could simply teleport it and then attempt to “move” it along the zero vector.

That said, continuous collision introduces some of its own problems. The biggest (for my purposes, anyway) is that it’s definitely more complicated to implement. “Sliding” means figuring out which obstacle would be hit first. You can do raycasting in the direction of movement and see what the ray hits first, though that’s imprecise and opens you up to new kinds of edge cases. If you’re lucky, you’re using something like Unity and can cast the entire shape as a single unit. Otherwise, well, you have to do a bunch of math to find everything in the swept path, then sort them in the order they’d be hit.

The other big problem is that it’s more work at runtime. With discrete collision, you only need to check for collisions in the new location. That only costs more time when a lot of objects are bunched together in one place, which is unlikely. With continuous collision, everything along the swept path needs to be examined, and that means that the faster an object moves, the more expensive its movement becomes.

So, not quite a golden bullet for the tunnelling problem. But that’s not a surprise; the only way to prevent tunnelling is to check for objects between the start and end positions.


Which, then, do I want to implement here?

For platforms without floating point (including the PICO-8 and Game Boy), there’s a third, hybrid option. If everything’s expressed with integers (or fixed point), then the universe has a Planck length: a minimum distance that every other distance must be an integral multiple of. You can thus fake continuous collision by doing repeated steps of discrete collision, one Planck length at a time. Objects will be collided with in the correct order, and you can simply stop at the first overlap.

Of course, this eats up a lot of time, since it involves doing collision detection numerous times per object per frame. So unless your Planck length is really big, I’m not sure it’s worth it.

Instead, I’m going to try for continuous collision. It’s closer to “correct” (whatever that means), and it’s what I did for all of my other games so far. It’s definitely harder, thornier, more complicated, and slower, but dammit I like it. It should also save me from encountering surprise bugs later on, which means I can write collision code once and then pretty much forget about it. Ideal.

Getting started

Star Anise is the only entity at the moment, so as a first pass, I’m only going to implement collision with the world.

World collision is much easier! Everything is laid out in a fixed grid, so I already know where the cells are. Finding potential overlaps is fairly simple, and best of all, I don’t need to sort anything to know what order the cells are in.

Right away, I find I have another decision to make. I would normally want to use vector math here — the motion is some distance in some direction, and hey, that’s a vector. But vectors take up twice as much space (read: twice as many registers), and a lot of vector operations rely on division or square roots which are non-trivial on this hardware.

With a great reluctant sigh, I thus commit to one more approximation, one made on 8-bit hardware since time immemorial. I won’t actually move in the direction of motion; instead, I’ll move along the x-axis, then move along the y-axis separately. Diagonal movement could theoretically cut across some corners (or be unable to fit through very tight gaps), but those are very minor and unlikely inconveniences. More importantly, this handwaving can’t allow any impossible motion.

I’ve already taken for granted that entities will all be axis-aligned rectangles. I’m definitely not dealing with slopes on a goddamn Game Boy. That was hard enough to do from scratch on a modern computer.

But I’m getting ahead of myself. First things first: you may recall that Star Anise’s movement is a bit of a hack. Pressing a direction button only adds to or subtracts from the sprite coordinates in the OAM buffer; his position isn’t actually stored in RAM anywhere. In fact, thanks to my slightly nonlinear storytelling across these posts, his movement isn’t stored anywhere either! The input-reading code writes directly to the OAM buffer. Whoops. I intended to fix that later, and now it’s later, so here we go.

1
2
3
4
5
; Somewhere in RAM, before anise_facing etc
anise_x:
    db
anise_y:
    db

So far, so good. OAM is populated in two places (and I should fix that later, too): once during setup, and once in the main game loop. Both will need to be updated to use these values.

Setup needs to initialize them first, of course:

1
2
3
4
    ld a, 64
    ld [anise_x], a
    ld [anise_y], a
    ; ... initialize anise_facing, etc ...

And now the OAM setup can be fixed. But, surprise! I left myself another hardcoded knot to untangle: even the relative positions of the sprites are hardcoded. Okay, so, those need to be put somewhere too. Eventually I’m going to need some kinda entity structure, but since there’s only one entity, I’ll just slap it into a constant somewhere.

(I guess my programming philosophy is leaking out a bit here. Don’t worry about structure until you need it, and you don’t need it until you need it twice. Once code works for one thing, it’s relatively straightforward to make it work for n things, and you have fewer things to worry about while you’re just trying to make something work.)

1
2
3
4
5
; In ROM somewhere
ANISE_SPRITE_POSITIONS:
    db -2, -20
    db -8, -14
    db 0, -14

It’s not immediately obvious from looking at these numbers, but I’m taking Star Anise’s position to mean the point on the ground between his feet. That’s the best approximation of where he is, after all.

(Early in game development, it seems natural to treat position as the upper-left corner of the sprite, so you can simply draw the sprite at the entity’s position — but that tangles the world model up with the sprite you happen to have at the moment. Imagine the havoc it’d wreak if you changed the size of the sprite later!)

Okay, now I can finally—

What? How does the code know there are exactly 3 sprites, on this byte-level platform? Because I’m hardcoding it. Shut up already I’ll fix it later

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
    ; Load the x and y coordinates into the b and c registers
    ld hl, anise_x
    ld b, [hl]
    inc hl
    ld c, [hl]
    ; Leave hl pointing at the sprite positions, which are
    ; ordered so that hl+ will step through them correctly
    ld hl, ANISE_SPRITE_POSITIONS

    ; ANTENNA
    ; x-coord
    ; The x coordinate needs to be added to the sprite offset,
    ; AND the built-in OAM offset (8, 16).  Reading the sprite
    ; offset first allows me to use hl+.
    ld a, [hl+]
    add a, b
    add a, 8
    ; Previously, hl pointed into the OAM buffer and advanced
    ; throughout this code, but now I'm using hl for something
    ; else, so I use direct addresses of positions within the
    ; buffer.  Obviously this is a kludge and won't work once
    ; I stop hardcoding sprites' positions in OAM, but, you
    ; know, I'll fix it later.
    ld [oam_buffer + 1], a
    ; y-coord
    ld a, [hl+]
    add a, c
    add a, 16
    ld [oam_buffer + 0], a
    ; This stuff is still hardcoded.
    ; chr index
    xor a
    ld [oam_buffer + 2], a
    ; attributes
    ld [oam_buffer + 3], a

    ; The rest of this is not surprising.

    ; LEFT PART
    ; x-coord
    ld a, [hl+]
    add a, b
    add a, 8
    ld [oam_buffer + 5], a
    ; y-coord
    ld a, [hl+]
    add a, c
    add a, 16
    ld [oam_buffer + 4], a
    ; chr index
    ld a, 2
    ld [oam_buffer + 6], a
    ; attributes
    ld a, %00000001
    ld [oam_buffer + 7], a

    ; RIGHT PART
    ; x-coord
    ld a, [hl+]
    add a, b
    add a, 8
    ld [oam_buffer + 9], a
    ; y-coord
    ld a, [hl+]
    add a, c
    add a, 16
    ld [oam_buffer + 8], a
    ; chr index
    ld a, 4
    ld [oam_buffer + 10], a
    ; attributes
    ld a, %00000001
    ld [oam_buffer + 11], a

Boot up the game, and… it looks the same! That’s going to be a running theme for a little bit here. Sorry, this isn’t a particularly screenshot-heavy post. It’s all gonna be math and code for a while.

Now I need to split apart the code that reads input and applies movement to OAM. Reading input gets much simpler, since it doesn’t have to do anything any more, just compute a dx and dy.

This code does still have looming questions, such as how to handle pressing two opposite directions (which is impossible on hardware but easy on an emulator), or whether diagonal movement should be fixed so that Anise doesn’t move at \(\sqrt{2}\) his movement speed.

Later. Seriously the actual code has so many XXX and TODO and FIXME comments that I edit out of these posts.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
    ; Anise update loop
    ; Stick dx and dy in the b and c registers.
    ld a, [buttons]
    ; b/c: dx/dy
    ld b, 0
    ld c, 0
    bit PADB_LEFT, a
    jr z, .skip_left
    dec b
.skip_left:
    bit PADB_RIGHT, a
    jr z, .skip_right
    inc b
.skip_right:
    bit PADB_UP, a
    jr z, .skip_up
    dec c
.skip_up:
    bit PADB_DOWN, a
    jr z, .skip_down
    inc c
.skip_down:

    ; For now just add b and c to Anise's coordinates.  This
    ; is where collision detection will go in a moment!
    ld a, [anise_x]
    add a, b
    ld [anise_x], a
    ld a, [anise_y]
    add a, c
    ld [anise_y], c

All that’s left is to more explicitly update the OAM buffer!

This code ends up looking fairly similar to the setup code. So similar, in fact, that I wonder if these blocks should be merged, but I’ll do that later:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
    ; Load x and y into b and c
    ld hl, anise_x
    ld b, [hl]
    inc hl
    ld c, [hl]
    ; Point hl at the sprite positions
    ld hl, ANISE_SPRITE_POSITIONS

    ; ANTENNA
    ; x-coord
    ld a, [hl+]
    add a, b
    add a, 8
    ld [oam_buffer + 1], a
    ; y-coord
    ld a, [hl+]
    add a, c
    add a, 16
    ld [oam_buffer + 0], a
    ; LEFT PART
    ; x-coord
    ld a, [hl+]
    add a, b
    add a, 8
    ld [oam_buffer + 5], a
    ; y-coord
    ld a, [hl+]
    add a, c
    add a, 16
    ld [oam_buffer + 4], a
    ; RIGHT PART
    ; x-coord
    ld a, [hl+]
    add a, b
    add a, 8
    ld [oam_buffer + 9], a
    ; y-coord
    ld a, [hl+]
    add a, c
    add a, 16
    ld [oam_buffer + 8], a

Phew! And the game plays exactly the same as before. Programming is so rewarding.

On to the main course!

Collision detection, sort of

So. First pass. Star Anise can only collide with the map.

Ah, but first, what size is Star Anise himself? I’ve only given him a position, not a hitbox. I could use his sprite as the hitbox, but with his helmet being much bigger than his body, that’ll make it seem like he can’t get closer than a foot to anything else. I’d prefer if he had an explicit radius.

1
2
3
; in ROM somewhere
ANISE_RADIUS:
    db 3

Remember, Star Anise’s position is the point between his feet. This describes his hitbox as a square, centered at that point, with sides 6 pixels long. The top and bottom edges of his hitbox are thus at y - r and y + r, which makes for some pleasing symmetry.

(Making hitboxes square doesn’t save a lot of effort or anything, but switching to rectangles later on wouldn’t be especially difficult either.)

The plan

My plan for moving rightwards, which I came up with after a lot of very careful and very messy sketching, looks like this:

  1. Figure out which rows I’m spanning.

  2. Move right until the next grid line. No new obstacle can possibly be encountered until then, so there’s nothing to check.

    (Unless I’m somehow already overlapping an obstacle, of course, but then I’d rather be able to move out of the obstacle than stay stuck and possibly softlock the game.)

  3. In the next grid column, check every cell that’s in a spanned row. If any of those cells block us, stop here. Otherwise, move to the next grid line (8 pixels).

  4. Repeat until I run out of movement.

    (It’s very unlikely the previous step would happen more than once; an entity would have to move more than 8 pixels per frame, which is 3 entire screen widths per second.)

Here’s a diagram. In this case, step 3 checks two cells for each column, but it might check more or fewer depending on how the entity is positioned. (It’ll never need to check more than one cell more than the entity’s height.)

Illustration of the above algorithm

Seems straightforward enough. But wait!

Edge case

I’ll save you a bunch of debugging anguish on my part and skip to the punchline: there’s an edge case.

I mean, literally, the case of when the entity’s edge is already against a grid line. That’ll happen fairly frequently — every time an entity collides with the map, it’ll naturally stop with its edge aligned to the grid.

The problem is all the way back in step 1. Remember, I said that to figure out which grid row or column a point belongs to, I need to divide by 8 (or shift right by 3). So the rows an entity spans must count from its top edge divided by 8, to its bottom edge divided by 8. Right?

Well…

Diagram showing division by 8 for several possible positions; when the bottom of the entity touches a grid line, it appears to be jutting into the row below

Everything’s fine until the entity’s bottom edge is exactly flush with the grid line, as in the last example. Then it seems to be jutting into the row below, even though no part of it is actually inside that row. If the entity tried to move rightwards from here, it might get blocked on something in row 1! Even worse, if row 1 were a solid wall that it had just run into, it wouldn’t be able to move left or right at all!

What happened here? There’s a hint in how I laid out the diagram.

There’s something akin to the fencepost problem here. I’ve been talking about rows and columns of the grid as if they were regions — “row 1” labels a rectangular strip of the world. But pixel coordinates don’t describe regions! They describe points. A pixel is a square area, but a pixel coordinate is the point at the upper left corner of that area.

In the incorrect example, the bottom of the entity is at y = 8, even though the row of pixels described by y = 8 doesn’t contain any part of the hitbox. I’m using the coordinate of the pixel’s top edge to describe a box’s bottom edge, and it falls apart when I try to reinterpret that coordinate as a region. In terms of area, y = 8 really names the first row of pixels that the entity doesn’t overlap.

To work around this, I need to adjust how I convert a coordinate to the corresponding grid cell, but only when that coordinate describes the right or bottom of a bounding box. Bottom pixel 8 should belong to row 0, but 9 should still end up in row 1.

As luck would have it, I’m using integers for coordinates, which means there’s a Planck length — a minimum distance of which all other distances are a multiple. That length is, of course, 1 pixel. If I subtract that length from a bottom coordinate, I get the next nearest coordinate going upwards. If the original coordinate was on a grid line, it’ll retreat back into the cell above; otherwise, it’ll stay in the same cell. You can check this with the diagram, if you need some convincing.

(This works for any fixed point system; integers are the special case of fixed point with zero fractional bits. It would not work so easily with floating point — subtracting the smallest possible float value will usually do nothing, because there’s not enough precision to express the difference. But then, if you have floating point, you probably have division and can write vector-based collision instead of taking grid-based shortcuts.)

All that is to say, I just need to subtract 1 before shifting. For clarity, I’ll write these as macros to convert a coordinate in a to a grid cell. I call the top or left conversion inclusive, because it includes the pixel the coordinate refers to; conversely, the bottom and right conversion is exclusive, like how a bottom of 8 actually excludes the pixels at y = 8.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
; Given a point on the top or left of a box, convert it to the
; containing grid cell.
ToInclusiveCell: MACRO
    ; This is just floor division
    srl a
    srl a
    srl a
ENDM
; Given a point on the bottom or right of a box, convert it to
; the containing grid cell.
ToExclusiveCell: MACRO
    ; Deal with the exclusive edge by subtracting the planck
    ; length, then flooring
    dec a
    srl a
    srl a
    srl a
ENDM

At last, I can write some damn code!

Some damn code

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
    ; Here, b and c contain dx and dy, the desired movement.

    ; First, figure out which columns we might collide with.
    ; The NEAREST is the first one to our right that we're not
    ; already overlapping, i.e. the one /after/ the one
    ; containing our right edge.  That's Exc(x + r) + 1.
    ; The FURTHEST is the column that /will/ contain our right
    ; edge.  That's Exc(x + r + dx).
    ld hl, ANISE_RADIUS
    ; Put the NEAREST column in d
    ld a, [anise_x]             ; a = x
    add a, [hl]                 ; a = x + r
    ld e, a                     ; e = x + r
    ToExclusiveCell
    inc a                       ; a = Exc(x + r) + 1
    ld d, a                     ; d = Exc(x + r) + 1
    ; Put the FURTHEST column in e
    ld a, e                     ; a = x + r
    add a, b                    ; a = x + r + dx
    ToExclusiveCell
    ld e, a                     ; e = Exc(x + r + dx)

    ; Loop over columns in [d, e].
    ; If d > e, this movement doesn't cross a grid line, so
    ; nothing can stop us and we can skip all this logic.
    ld a, e
    cp d
    jp c, .done_x
    ; We don't need dx for now, so stash bc for some work space
    push bc
.x_row_scan:
    ; For each column we might cross: check whether any of the
    ; rows we span will block us.
    ; Hm.  This code probably should've been outside the loop.
    ld a, [anise_y]
    ld hl, ANISE_RADIUS
    sub a, [hl]
    ToInclusiveCell
    ld b, a                     ; b = minimum y
    ld a, [anise_y]
    add a, [hl]
    ToExclusiveCell
    ld c, a                     ; c = maximum/current y

.x_column_scan:
    ; Put the cell's row and column in bc, and call a function
    ; to check its "map flags".  I'll define that in a moment,
    ; but for now I'll assume that if bit 0 is set, that means
    ; the cell is solid.
    ; This is also why the inner loop counts down with c, not
    ; up with b: get_cell_flags wants the y coord in c, and
    ; this way, it's already there!
    push bc
    ld b, d
    call get_cell_flags
    pop bc
    ; If this produces zero, we can skip ahead
    and a, $01
    jr z, .not_blocked

    ; We're blocked!  Stop here.  Set x so that we're butted
    ; against this cell, which means subtract our radius from
    ; its x coordinate.
    ; Note that this can't possibly move us further than dx,
    ; because dx was /supposed/ to move us INTO this cell.
    ld a, d
    ; This is a /left/ shift three times, for cell -> pixel
    sla a
    sla a
    sla a
    sub a, [hl]
    ld [anise_x], a
    ; Somewhat confusing pop, to restore dx and dy.
    pop bc
    jp .done_x

.not_blocked:
    ; Not blocked, so loop to the next cell in this column
    dec c
    ld a, c
    cp b
    jr nc, .x_column_scan

    ; Finished checking one column successfully, so continue on
    ; to the next one
    inc d
    ld a, e
    cp d
    jr nc, .x_row_scan

    ; Done, and we never hit anything!  Update our position to
    ; what was requested
    pop bc
    ld a, [anise_x]
    add a, b
    ld [anise_x], a

I’ve also gotta implement get_cell_flags, which is slightly uglier than I anticipated.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
; Fetches properties for the map cell at the given coordinates.
; In: bc = x/y coordinates
; Out: a = flags
get_cell_flags:
    push hl
    push de
    ; I have to figure out what char is at these coordinates,
    ; which means consulting the map, which means doing math.
    ; The map is currently 16 (big) tiles wide, or 32 chars,
    ; so the byte for the indicated char is at b + 32 * c.
    ld hl, TEST_MAP_1
    ; Add x coordinate.  hl is 16 bits, so extend b to 16 bits
    ; using the d and e registers separately, then add.
    ld d, 0
    ld e, b
    add hl, de
    ; Add y coordinate, with stride of 32, which we can do
    ; without multiplying by shifting left 5.  Alas, there are
    ; no 16-bit shifts, so I have to do this by hand.
    ; First get the 5 high bits by copying y into d, then
    ; shifting the 3 low bits off the right end.
    ld d, c
    srl d
    srl d
    srl d
    ; Then get the low 3 bits into the high 3 by swapping,
    ; shifting, and masking them off.
    ld a, c
    swap a
    sla a
    and a, $e0
    ld e, a
    ; Not sure that was really any faster than just shifting
    ; left through the carry flag 5 times.  Oh well.  Add.
    add hl, de

    ; At last, we know the char.  I don't have real flags at
    ; the moment, so I just hardcoded the four chars that make
    ; up the small rock tile.
    ld a, [hl]
    cp a, 2
    jr z, .blocking
    cp a, 3
    jr z, .blocking
    cp a, 12
    jr z, .blocking
    cp a, 13
    jr z, .blocking
    jr .not_blocking
    ; The rest should not be too surprising.
.blocking:
    ld a, 1
    jr .done
.not_blocking:
    xor a
.done:
    pop de
    pop hl
    ret

And that’s it!

That’s not it

The code I wrote only applies when moving right. It doesn’t handle moving left at all.

And here I run into a downside of continuous collision, at least in this particular case. Because of the special behavior of right/bottom edges, I can’t simply flip a sign to make this code work for leftwards movement as well. For example, the set of columns I might cross going rightwards is calculated exclusively, because my right edge is the one in front… but if I’m moving leftwards, it’s calculated inclusively. Those columns are also in reverse order and thus need iterating over backwards, so an inc somewhere becomes a dec, and so on.

I have two uncomfortable options for handling this. One is to add all the required conditional tests and jumps, but that adds a decent CPU cost to code that’s fairly small and potentially very hot, and complicates code that’s a bit dense and delicate to begin with. The other option is to copy-paste the whole shebang and adjust it as needed to go leftwards.

Guess which I did!

1
2
3
4
5
6
7
8
9
    ld a, b
    cp a, $80
    jp nc, .negative_x
.positive_x:
    ; ... everything above ...
    jp .done_x
.negative_x:
    ; ... everything above, flipped ...
.done_x:

Ugh. Don’t worry, though — it gets worse later on!

I could copy-paste for y movement too and give myself a total of four blocks of similar code, but I’ll hold off on that for now.

Ah.

You want the payoff, don’t you.

Well, I’m warning you now: the next post gets much hairier, and if I show you a GIF now, there won’t be any payoff next time.

You sure? Really?

No going back!

Star Anise walking around, but not through a rock!

I admit, this was pretty damn satisfying the first time it actually worked. Collision detection is a pain in the ass, but it’s the first step to making a game feel like a game. Games are about working within limitations, after all!

An aside: debugging

I’ve made this adventure seem much easier than it actually was by eliding all the mistakes. I made a lot of mistakes, and as I said upfront, it can be very difficult to notice heisenbugs or figure out exactly what’s causing them.

One thing that helped tremendously near the beginning was to hack Star Anise to have a fourth sprite: a solid black 6×6 square under his feet. That let me see where he was actually supposed to be able to stand. Highly recommend it. All I did was copy/paste everywhere that mentioned his sprites to add a fourth one, and position it centered under his feet.

(On any other system, I’d just draw collision rectangles everywhere, but the Game Boy is sprite-based so that’s not really gonna fly.)

I also had pretty good success with writing intermediate values to unused bytes in RAM, so I could inspect them in mGBA’s memory viewer even after the movement was finished. And of course, as an absolute last resort, bgb has an interactive graphical debugger. (Nothing against bgb per se; I just prefer not to rely on closed-source software running in Wine if I can at all get away with it.)

To be continued

Obviously, this isn’t anywhere near done. There’s no concept of collision with other entities, and before that’s even a possibility, I need a concept of other entities. I left myself a long trail of do-it-laters. There are even risks of overflow and underflow in a couple places, which I didn’t bother pointing out because I completely overhaul this code later.

But it’s a big step forward, and now I just need a few more big steps forward. (I say, four months later, long after all those steps are done.)

I already have some future ideas in mind, like: what if a map tile weren’t completely solid, but had its own radius? Could I implement corner cutting, where the game gently guides you if you get stuck on a corner by only a single pixel? What about having tiles that are 45° angles, just to cut down on the overt squareness of the map?

Well. Maybe, you know, later.

Anyway, that brings us up to commit da7478e. It’s all downhill from here.

Next time: more collision detection, and fixed-point arithmetic!

Cheezball Rising: Opening a dialogue

Post Syndicated from Eevee original https://eev.ee/blog/2018/10/09/cheezball-rising-opening-a-dialogue/

This is a series about Star Anise Chronicles: Cheezball Rising, an expansive adventure game about my cat for the Game Boy Color. Follow along as I struggle to make something with this bleeding-edge console!

GitHub has intermittent prebuilt ROMs, or you can get them a week early on Patreon if you pledge $4. More details in the README!


In this issue, I draw some text!

Previously: I get a Game Boy to meow.
Next: collision detection, ohh nooo

Recap

The previous episode was a diversion (and left an open problem that I only solved after writing it), so the actual state of the game is unchanged.

Star Anise walking around a moon environment in-game, animated in all four directions

Where should I actually go from here? Collision detection is an obvious place, but that’s hard. Let’s start with something a little easier: displaying scrolling dialogue text. This is likely to be a dialogue-heavy game, so I might as well get started on that now.

Planning

On any other platform, I’d dive right into it: draw a box on the screen somewhere, fill it with text.

On the Game Boy, it’s not quite that simple. I can’t just write text to the screen; I can only place tiles and sprites.

Let’s look at how, say, Pokémon Yellow handles its menu.

Pokémon Yellow with several levels of menu open

This looks — feels — like it’s being drawn on top of the map, and that sub-menus open on top of other menus. But it’s all an illusion! There’s no “on top” here. This is a completely flat image made up of tiles, like anything else.

The same screenshot, scaled up, with a grid showing the edges of tiles

This is why Pokémon has such a conspicuously blocky font: all the glyphs are drawn to fit in a single 8×8 char, so “drawing” text is as simple as mapping letters to char indexes and drawing them onto the background. The map and the menu are all on the same layer, and the game simply redraws whatever was underneath when you close something. Part of the illusion is that the game is clever enough to hide any sprites that would overlap the menu — because sprites would draw on top! (The Game Boy Color has some twiddles for controlling this layering, but Yellow was originally designed for the monochrome Game Boy.)

A critical reason that this actually works is that in Pokémon, the camera is always aligned to the grid. It scrolls smoothly while you’re walking, but you can’t actually open the menu (or pick up an item, or talk to someone, or do anything else that might show text) until you’ve stopped moving. If you could, the menu would be misaligned, because it’s part of the same grid as the map!

This poses a slight problem for my game. Star Anise isn’t locked to the grid like the Pokémon protagonist is, and unlike Link’s Awakening, I do want to have areas larger than the screen that can scroll around freely.

I know offhand that there are a couple ways to do this. One is the window, an optional extra opaque layer that draws on top of the background, with its top-left corner anchored to any point on the screen. Another is to change some display registers in the middle of the screen redrawing. If you’re thinking of any games with a status bar at the bottom or right, chances are they use the window; games with a status bar at the top have to use display register tricks.

But I don’t want to worry about any of this right now, before I even have text drawing. I know it’s possible, so I’ll deal with it later. For now, drawing directly onto the background is good enough.

Font decisions

Let’s get back to the font itself. I’m not in love with the 8×8 aesthetic; what are my other options? I do like the text in Oracle of Ages, so let’s have a look at that:

Oracle of Ages, also scaled up with a grid, showing its taller text

Ah, this is the same approach again, except that letters are now allowed to peek up into the char above. So these are 8×16, but the letters all occupy a box that’s more like 6×9, offering much more familiar proportions. Oracle of Ages is designed for the Game Boy Color, which has twice as much char storage space, so it makes sense that they’d take advantage of it for text like this.

It’s not bad, but the space it affords is still fairly… limited. Only 16 letters will fit in a line, just as with Pokémon, and that means a lot of carefully wording things to be short and use mostly short words as well. That’s not gonna cut it for the amount of dialogue I expect to have.

(You may be wondering, as I did, how Oracle pulled off this grid-aligned textbox. In small buildings and the overworld, each room is exactly the size of the screen, so there’s no scrolling and no worry about misaligned text. But how does the game handle showing text inside a dungeon, where a room is bigger than the screen and can scroll freely? The answer is: it doesn’t! The textbox is just placed as close as possible to the position shown in this screenshot, so the edges might be misaligned by up to 4 pixels. In 20 years, I never noticed this until I thought to check how they were handling it. I’m sure there’s a lesson, here.)

What other options do I have? It seems like I’m limited to multiples of 8 here, surely. (The answer may be obvious to some of you, but shh, don’t read ahead.)

The answer lies in the very last game released for the Game Boy Color: Harry Potter and the Chamber of Secrets. Whatever deep secrets were learned during the Game Boy’s lifetime will surely be encapsulated within this, er, movie tie-in game.

Harry Potter and the Chamber of Secrets, also scaled up with a grid, showing its text isn't fixed to the grid

Hot damn. That is a ton of text in a relatively small amount of space! And it doesn’t fit the grid! How did they do that?

The answer is… exactly how you’d think!

Tile display for the above screenshot, showing that the text is simply written across consecutive tiles

With a fixed-width font like in Pokémon and Zelda games, the entire character set is stored in VRAM, and text is drawn by drawing a string of characters. With a variable-width font like in Harry Potter, a block of VRAM is reserved for text, and text is drawn into those chars, in software. Essentially, some chars are used like a canvas and have text rendered to them on the fly. The contents of the background layer might look like this in the two cases:

Illustration of fixed width versus variable width text

Some pros of this approach:

  • Since the number of chars required is constant and the font is never loaded directly into char memory, the font can have arbitrarily many glyphs in it. Multiple fonts could be used at the same time, even. (Of course, if you have more than 256 glyphs, you’ll have to come up with a multi-byte encoding for actually storing the text…)

  • A lot more text can fit in one line while still remaining readable.

  • It has the potential to look very cool. I definitely want to squeeze every last drop of fancy-pants graphical stuff that I can from this hardware.

And, cons:

  • It’s definitely more complicated! But I only have to write the code once, and since the game won’t be doing anything but drawing dialogue while the box is up, I don’t think I’ll be in danger of blowing my CPU budget.

  • Colored text becomes a bit trickier. But still possible, so, we can worry about that later.

  • Fixed text that doesn’t scroll, like on menus and whatnot, will be something of a problem — this whole idea relies on amortizing the text rendering across multiple frames. On the other hand, this game shouldn’t have too much of that, and this sounds like a good excuse to hand-draw fixed text (which can then be much more visually interesting). At worst, I could just render the fixed text ahead of time.

Well, I’m sold. Let’s give it a shot.

First pass

Well, I want to do something on a button press, so, let’s do that.

A lot of games (older ones especially) have bugs from switching “modes” in the same frame that something else happens. I don’t entirely understand why that’s so common and should probably ask some speedrunners, but I should be fine if I do mode-switching first thing in the frame, and then start over a new frame when switching back to “world” mode. Right? Sure.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
    ; ... button reading code in main loop ...
    bit BUTTON_A, a
    jp nz, .do_show_dialogue

    ; ... main loop ...

    ; Loop again when done
    jp vblank_loop

.do_show_dialogue:
    call show_dialogue
    jp vblank_loop

The extra level of indirection added by .do_show_dialogue is just so the dialogue code itself isn’t responsible for knowing where the main loop point is; it can just ret.

Now to actually do something. This is a first pass, so I want to do as little as possible. I’ll definitely need a palette for drawing the text — and here I’m cutting into my 8-palette budget again, which I don’t love, but I can figure that out later. (Maybe with some shenanigans involving changing the palettes mid-redraw, even.)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
PALETTE_TEXT:
    ; Black background, white text...  then gray shadow, maybe?
    dcolor $000000
    dcolor $ffffff
    dcolor $999999
    dcolor $666666

show_dialogue:
    ; Have to disable the LCD to do video work.  Later I can do
    ; a less jarring transition
    DisableLCD

    ; Copy the palette into slot 7 for now
    ld a, %10111000
    ld [rBCPS], a
    ld hl, PALETTE_TEXT
    REPT 8
    ld a, [hl+]
    ld [rBCPD], a
    ENDR

I also know ahead of time what chars will need to go where on the screen, so I can fill them in now.

Note that I really ought to blank them all out, especially since they may still contain text from some previous dialogue, but I don’t do that yet.

An obvious question is: which tiles? I think I said before that with 512 chars available, and ¾ of those still being enough to cover the entire screen in unique chars, I’m okay with dedicating a quarter of my space to UI stuff, including text. To keep that stuff “out of the way”, I’ll put them at the “end” — bank 1, starting from $80.

I’m thinking of having characters be about the same proportions as in the Oracle games. Those games use 5 rows of tiles, like this:

1
2
3
4
5
top of line 1
bottom of line 1
top of line 2
bottom of line 2
blank

Since the font is aligned to the bottom and only peeks a little bit into the top char, the very top row is mostly blank, and that serves as a top margin. The bottom row is explicitly blank for a bottom margin that’s nearly the same size. The space at the top of line 2 then works as line spacing.

I’m not fixed to the grid, so I can control line spacing a little more explicitly. But I’ll get to that later and do something really simple for now, where $ff is a blank tile:

1
2
3
4
5
6
7
8
9
+--+--+--+--+--+--+--+--+--+--+--+--+--+---+
|ff|ff|ff|ff|ff|ff|ff|ff|ff|ff|ff|ff|ff|...|
+--+--+--+--+--+--+--+--+--+--+--+--+--+---+
|ff|80|82|84|86|88|8a|8c|8e|90|92|94|96|...|
+--+--+--+--+--+--+--+--+--+--+--+--+--+---+
|ff|81|83|85|87|89|8b|8d|8f|91|93|95|97|...|
+--+--+--+--+--+--+--+--+--+--+--+--+--+---+
|ff|ff|ff|ff|ff|ff|ff|ff|ff|ff|ff|ff|ff|...|
+--+--+--+--+--+--+--+--+--+--+--+--+--+---+

This gives me a canvas for drawing a single line of text. The staggering means that the first letter will draw to adjacent chars $80 and $81, rather than distant cousins like $80 and $a0.

You may notice that the below code updates chars across the entire width of the grid, not merely the screen. There’s not really any good reason for that.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
    ; Fill text rows with tiles (blank border, custom tiles)
    ; The screen has 144/8 = 18 rows, so skip the first 14 rows
    ld hl, $9800 + 32 * 14
    ; Top row, all tile 255
    ld a, 255
    ld c, 32
.loop1:
    ld [hl+], a
    dec c
    jr nz, .loop1

    ; Text row 1: 255 on the edges, then middle goes 128, 130, ...
    ld a, 255
    ld [hl+], a
    ld a, 128
    ld c, 30
.loop2:
    ld [hl+], a
    add a, 2
    dec c
    jr nz, .loop2
    ld a, 255
    ld [hl+], a

    ; Text row 2: same as above, but middle is 129, 131, ...
    ld a, 255
    ld [hl+], a
    ld a, 129
    ld c, 30
.loop3:
    ld [hl+], a
    add a, 2
    dec c
    jr nz, .loop3
    ld a, 255
    ld [hl+], a

    ; Bottom row, all tile 255
    ld a, 255
    ld c, 32
.loop4:
    ld [hl+], a
    dec c
    jr nz, .loop4

Now I need to repeat all of that, but in bank 1, to specify the char bank (1) and palette (7) for the corresponding tiles. Those are the same for the entire dialogue box, though, so this part is easier.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
    ; Switch to VRAM bank 1
    ld a, 1
    ldh [rVBK], a

    ld a, %00001111  ; bank 1, palette 7
    ld hl, $9800 + 32 * 14
    ld c, 32 * 4  ; 4 rows
.loop5:
    ld [hl+], a
    dec c
    jr nz, .loop5

    EnableLCD

Time to get some real work done. Which raises the question: how do I actually do this?

If you recall, each 8-pixel row of a char is stored in two bytes. The two-bit palette index for each pixel is split across the corresponding bit in each byte. If the leftmost pixel is palette index 01, then bit 7 in the first byte will be 0, and bit 7 in the second byte will be 1.

Now, a blank char is all zeroes. To write a (left-aligned) glyph into a blank char, all I need to do is… well, I could overwrite it, but I could just as well OR it. To write a second glyph into the unused space, all I need to do is shift it right by the width of the space used so far, and OR it on top. The unusual split layout of the palette data is actually handy here, because it means the size of the shift matches the number of pixels, and I don’t have to worry about overflow.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
0 0 0 0 0 0 0 0  <- blank glyph

1 1 1 1 0 0 0 0  <- some byte from the first glyph
↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
1 1 1 1 0 0 0 0  <- ORed together to display first character

          1 1 1 1 0 0 0 0  <- some byte from the second glyph,
                              shifted by 4 (plus a kerning pixel)
↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
1 1 1 1 0 1 1 1  <- ORed together to display first two characters

The obvious question is, well, what happens to the bits from the second character that didn’t fit? I’ll worry about that a bit later.

Oh, and finally, I’ll need a font, plus some text to display. This is still just a proof of concept, so I’ll add in a couple glyphs by hand.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
; somewhere in ROM
font:
; A
    ; First byte indicates the width of the glyph, which I need
    ; to know because the width varies!
    db 6
    dw `00000000
    dw `00000000
    dw `01110000
    dw `10001000
    dw `10001000
    dw `10001000
    dw `11111000
    dw `10001000
    dw `10001000
    dw `10001000
    dw `10001000
    dw `00000000
    dw `00000000
    dw `00000000
    dw `00000000
    dw `00000000
; B
    db 6
    dw `00000000
    dw `00000000
    dw `11110000
    dw `10001000
    dw `10001000
    dw `10001000
    dw `11110000
    dw `10001000
    dw `10001000
    dw `10001000
    dw `11110000
    dw `00000000
    dw `00000000
    dw `00000000
    dw `00000000
    dw `00000000

text:
    ; Shakespeare it ain't.
    ; Need to end with a NUL here so I know where the text
    ; ends.  This isn't C, there's no automatic termination!
    db "ABABAAA", 0

And here we go!

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
    ; ----------------------------------------------------------
    ; Setup done!  Real work begins here
    ; b: x-offset within current tile
    ; de: text cursor + current character tiles
    ; hl: current VRAM tile being drawn into
    ld b, 0
    ld de, text
    ld hl, $8800

    ; This loop waits for the next vblank, then draws a letter.
    ; Text thus displays at ~60 characters per second.
.next_letter:
    ; This is probably way more LCD disabling than is strictly
    ; necessary, but I don't want to worry about it yet
    EnableLCD
    call wait_for_vblank
    DisableLCD

    ld a, [de]                  ; get current character
    and a                       ; if NUL, we're done!
    jr z, .done
    inc de                      ; otherwise, increment

    ; Get the glyph from the font, which means computing
    ; font + 33 * a.
    ; A little register juggling.  hl points to the current
    ; char in VRAM being drawn to, but I can only do a 16-bit
    ; add into hl.  de I don't need until the next loop,
    ; since I already read from it.  So I'm going to push de
    ; AND hl, compute the glyph address in hl, put it in de,
    ; then restore hl.
    push de
    push hl
    ; The text is written in ASCII, but the glyphs start at 0
    sub a, 65
    ld hl, font
    ld de, 33                   ; 1 width byte + 16 * 2 tiles
    ; This could probably be faster with long multiplication
    and a
.letter_stride:
    jr z, .skip_letter_stride
    add hl, de
    dec a
    jr .letter_stride
.skip_letter_stride:
    ; Move the glyph address into de, and restore hl
    ld d, h
    ld e, l
    pop hl

    ; Read the first byte, which is the character width.  This
    ; overwrites the character, but I have the glyph address,
    ; so I don't need it any more
    ld a, [de]
    inc de

    ; Copy into current chars
    ; Part 1: Copy the left part into the current chars
    push af                     ; stash width
    ; A glyph is two chars or 32 bytes, so row_copy 32 times
    ld c, 32
    ; b is the next x position we're free to write to.
    ; Incrementing it here makes the inner loop simpler, since
    ; it can't be zero.  But it also means two jumps per loop,
    ; so, ultimately this was a pretty silly idea.
    inc b
.row_copy:
    ld a, [de]                  ; read next row of character

    ; Shift right by b places with an inner loop
    push bc                     ; preserve b while shifting
    dec b
.shift:                         ; shift right by b bits
    jr z, .done_shift
    srl a
    dec b
    jr .shift
.done_shift:
    pop bc

    ; Write the updated byte to VRAM
    or a, [hl]                  ; OR with current tile
    ld [hl+], a
    inc de
    dec c
    jr nz, .row_copy
    pop af                      ; restore width

    ; Part 2: Copy whatever's left into the next char
    ; TODO  :)

    ; Cleanup for next iteration
    ; Undo the b increment from way above
    dec b
    ; It's possible I overflowed into the next column, in which
    ; case I want to leave hl where it is: pointing at the next
    ; column.  Otherwise, I need to back it up to where it was.
    ; Of course, I also need to update b, the x offset.
    add a, b                    ; a <- new x offset
    ; If the new x offset is 8 or more, that's actually the next
    ; column
    cp a, 8
    jr nc, .wrap_to_next_tile
    ld bc, -32                  ; a < 8: back hl up
    add hl, bc
    jr .done_wrap
.wrap_to_next_tile:
    sub a, 8                    ; a >= 8: subtract tile width
    ld b, a
.done_wrap:
    ; Either way, store the new x offset into b
    ld b, a

    ; And loop!
    pop de                      ; pop text pointer
    jr .next_letter

.done:
    ; Undo any goofy stuff I did, and get outta here
    EnableLCD
    ; Remember to reset bank to 0!
    xor a
    ldh [rVBK], a
    ret

Phew! That was a lot, but hopefully it wasn’t too bad. I hit a few minor stumbling blocks, but as I recall, most of them were of the “I get the conditions backwards every single time I use cp augh” flavor. (In fact, if you look at the actual commit the above is based on, you may notice that I had the condition at the very end mixed up! It’s a miracle it managed to print part of the second letter at all.)

There are a lot of caveats in this first pass, including that there’s nothing to erase the dialogue box and reshow the map underneath it. (But I might end up using the window for this anyway, so there’s no need for that.)

As a proof of concept, though, it’s a great start!

Screenshot of Anise, with a black dialogue box that says: A|

That’s the letter A, followed by the first two pixels of the letter B. I didn’t implement the part where letters spill into the next column, yet.

Guess I’d better do that!

Second pass

One of the big problems with the first pass was that I had to turn the screen off to do the actual work safely. Shifting a bunch of bytes by some amount is a little slow, since I can only shift one bit at a time and have to do it within a loop, and vblank only lasts for about 6.5% of the entire duration of the frame. If I continued like this, the screen would constantly flicker on and off every time I drew a new letter. Yikes.

I’ll solve this the same way I solve pretty much any other vblank problem: do the actual work into a buffer, then just copy that buffer during vblank. Since I intend to draw no more than one character per frame, and each character glyph is no wider than a single char column, I only need a buffer big enough to span two columns. Text covers two rows, also, so that’s four tiles total.

I also need to zero out the tile buffer when I first start drawing text — otherwise it may still have garbage left over from the last time text was displayed! — and this seems like a great opportunity to introduce a little fill function. Maybe then I’ll do the right damn thing and clear out other stuff on startup.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
; Utility code section

; fill c bytes starting at hl with a
; NOTE: c must not be zero
fill:
    ld [hl+], a
    dec c
    jr nz, fill
    ret

; ...

; Stick this at a fixed nice address for now, just so it's easy
; for me to look at and debug
SECTION "Text buffer", WRAM0[$C200]
text_buffer:
    ; Text is up to 8x16 but may span two columns, so carve out
    ; enough space for four tiles
    ds $40

show_dialogue:
    DisableLCD
    ; ... setup stuff ...
    EnableLCD

    ; Zero out the tile buffer
    xor a
    ld hl, text_buffer
    ld c, $40
    call fill

That first round of disabling and enabling the LCD is still necessary, because the setup work takes a little time, but I can get rid of that later too. For now, the priority is fixing the text scroll (and supporting text that spans more than one tile).

The code is the same up until I start copying the glyph into the tiles. Now it doesn’t go to VRAM, but into the buffer.

There’s another change here, too. Previously, I shifted the glyph right, letting bits fall off the right end and disappear. But the bits that drop off the end are exactly the bits that I need to draw to the next char. I could do a left shift to retrieve them, but I had a different idea: rotate the glyph instead.

Say I want to draw a glyph offset by 3 pixels. Then I want to do this:

1
2
3
4
5
6
7
8
abcdefgh  <- original glyph bits
fghabcde  <- rotate right 3
00011111  <- mask, which is just $ff shifted right 3

000abcde  <- rotated glyph AND mask gives the left part

11100000  <- mask, inverted
fgh00000  <- rotated glyph AND inverted mask gives the right part

The time and code savings aren’t huge, exactly, and nothing else is going on while text is rendering so it’s not like time is at a premium here. But hey this feels clever so let’s do it.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
    ; Copy into current chars
    push af                     ; stash width
    ld c, 32                    ; 32 bytes per row
    ld hl, text_buffer          ; new!
    ; This is still silly.
    inc b
.row_copy:
    ld a, [de]                  ; read next row of character
    ; Rotate right by b - 1 pixels -- remember, b contains the
    ; x-offset within the current tile where to start drawing
    push bc                     ; preserve b while shifting
    ld c, $ff                   ; initialize the mask
    dec b
    jr z, .skip_rotate
.rotate:
    ; Rotate the glyph (a), but shift the mask (c), so that the
    ; left end of the mask fills up with zeroes
    rrca
    srl c
    dec b
    jr nz, .rotate
.skip_rotate:
    push af                     ; preserve glyph
    and a, c                    ; mask right pixels
    ; Draw to left half of text buffer
    or a, [hl]                  ; OR with current tile
    ld [hl+], a
    ; Write the remaining bits to right half
    ld a, c                     ; put mask in a...
    cpl                         ; ...to invert it
    ld c, a                     ; then put it back
    pop af                      ; restore unmasked glyph
    and a, c                    ; mask left pixels
    ld [hl+], a                 ; and store them!
    ; Clean up after myself, and loop to the next row
    inc de                      ; next row of glyph
    pop bc                      ; restore counter!
    dec c
    jr nz, .row_copy
    pop af                      ; restore width

The use of the stack is a little confusing (and don’t worry, it only gets worse in later posts). Note for example that c is used as the loop counter, but since I don’t actually need its value within the body of the loop, I can push it right at the beginning and use c to hold the mask, then pop the loop counter back into place at the end.

This is where I first started to feel register pressure, especially when addresses eat up two of them. My options are pretty limited: I can store stuff on the stack, or store stuff in RAM. The stack is arguably harder to follow (and easier to fuck up, which I’ve done several times), but either way there’s the register ambiguity.

Which is shorter/faster? Well:

  • A push/pop pair takes 2 bytes and 7 cycles.

  • Immediate writing to RAM and immediate reading back from it takes 6 bytes and 8 cycles, and can only be done with a, so I’d probably have to copy into and out of some other register too.

  • Putting an address in hl, writing to it, then reading from it takes 5 bytes and 7 cycles, but requires that I can preserve hl. (On the other hand, if I can preserve the value of hl across a loop or something, then it’s amortized away and the read/write is only 2 bytes and 3 cycles. But if that’s the case, chances are that I’m not under enough register pressure to need using RAM in the first place.)

  • Parts of high RAM ($ff80 and up) are available for program use, and they can be read or written with the same instructions that operate on the control knobs starting at $ff00. A high RAM read and write takes 4 bytes and 6 cycles, which isn’t too bad, but once again I have to go through the a register so I’ll probably need some other copies.

Stack it is, then.

Anyway! Where were we. I need to now copy the buffer into VRAM.

You may have noticed that the buffer isn’t quite populated in char format. Instead, it’s populated like one big 16-pixel char, with the first 16 bits corresponding to the 16 pixels spanning both columns. VRAM, of course, expects to get all the pixels from the first column, then all the pixels from the second column. If that’s not clear, here’s what I have (where the bits are in order from left to right, top to bottom):

1
2
3
AAAAAAAA BBBBBBBB  <- high bits for first row of pixels
aaaaaaaa bbbbbbbb  <- low bits for first row of pixels
... other rows ...

And here’s what I need to put in VRAM:

1
2
3
4
5
6
AAAAAAAA  <- high bits for first row of left column of pixels
aaaaaaaa  <- low bits for first row of left column of pixels
... other rows of left column ...
BBBBBBBB  <- high bits for first row of right column of pixels
bbbbbbbb  <- low bits for first row of right column of pixels
... other rows of right column ...

I hope that makes sense! To fix this, I use two loops (one for each column), and in each loop I copy every other byte into VRAM. That deinterlaces the buffer.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
    ; Draw the buffered tiles to vram
    ; The text buffer is treated like it's 16 pixels wide, but
    ; VRAM is of course only 8 pixels wide, so we need to do
    ; this in two iterations: the left two tiles, then the right
    pop hl                      ; restore hl (VRAM)
    push af                     ; stash width, again
    call wait_for_vblank        ; always wait before drawing
    push bc
    push de
    ; Draw the left two tiles
    ld c, $20
    ld de, text_buffer
.draw_left:
    ld a, [de]
    ; This double inc fixes the interlacing
    inc de
    inc de
    ld [hl+], a
    dec c
    jr nz, .draw_left
    ; Draw the right two tiles
    ld c, $20
    ; This time, start from the SECOND byte, which will grab
    ; all the bytes skipped by the previous loop
    ld de, text_buffer + 1
.draw_right:
    ld a, [de]
    inc de
    inc de
    ld [hl+], a
    dec c
    jr nz, .draw_right
    pop de
    pop bc
    pop af                      ; restore width, again

Just about done! There’s one last thing to do before looping to the next character. If this character did in fact span both columns, then the buffer needs to be moved to the left by one column. Here’s a simplified diagram, pretending chars are 5×5 and I just drew a B:

1
2
3
4
5
6
7
+-----+-----+.....+
| A  B|B    |     .
|A A B| B   |     .
|AAA B|B    |     .
|A A B| B   |     .
|A A B|B    |     .
+-----+-----+.....+

The left column is completely full, so I don’t need to buffer it any more. The next character wants to draw in the last partially full column, which here is the one containing the B; it’ll also want an empty right column to overflow into if necessary.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
    ; Increment the pixel offset and deal with overflow
    add a, b                    ; a <- new x offset
    ; Regardless of whether this glyph overflowed, the VRAM
    ; pointer was left at the beginning of the next (empty)
    ; column, and it needs rewinding to the right column
    ld bc, -32                  ; move the VRAM pointer back...
    add hl, bc                  ; ...to the start of the char
    cp a, 8
    jr nc, .wrap_to_next_char
    ; The new offset is less than 8, so this character didn't
    ; actually draw anything in the right column.  Move the
    ; VRAM pointer back a second time, to the left column,
    ; which still has space left
    add hl, bc
    jr .done_wrap
.wrap_to_next_char:
    ; The new offset is 8 or more, so this character drew into
    ; the next char.  Subtract 8, but also shift the text buffer
    ; by copying all the "right" chars over the "left" chars
    sub a, 8                    ; a >= 8: subtract char width
    push hl
    push af
    ; The easy way to do this is to walk backwards through the
    ; buffer.  This leaves garbage in the right column, but
    ; that's okay -- it gets overwritten in the next loop,
    ; before the buffer is copied into VRAM.
    ld hl, text_buffer + $40 - 1
    ld c, $20
.shift_buffer:
    ld a, [hl-]
    ld [hl-], a
    dec c
    jr nz, .shift_buffer
    pop af
    pop hl
.done_wrap:
    ld b, a                     ; either way, store into b

    ; Loop
    pop de                      ; pop text pointer
    jp .next_letter

And the test run:

Screenshot of Anise, with a black dialogue box that says: ABABAAA

Hey hey, success!

Quick diversion: Anise corruption

I didn’t mention it above because I didn’t actually use it yet, but while doing that second pass, I split the button-polling code out into its own function, read_input. I thought I might need it in dialogue as well (which has its own vblank loop and thus needs to do its own polling), but I didn’t get that far yet, so it’s still only called from the main loop.

While testing out the dialogue, I notice a teeny tiny problem.

A screenshot similar to the above, but with some mild graphical corruption on Anise

Well, yes, obviously there’s the problem of the textbox drawing underneath the player. Which is mostly a problem because the textbox doesn’t go away, ever. I’ll worry about that later.

The other problem is that Anise’s sprite is corrupt. Again. Argh!

A little investigation suggests that, once again, I’m blowing my vblank budget. But this time, it’s a little more reasonable. Remember, I’m overwriting Anise’s sprite after handling movement. That means I do a bunch of logic followed by writing to char data. No wonder there’s a problem. I must’ve just slightly overrun vblank when I split out read_input (or checked for the dialogue button press in the first place?), since call has a teeny tiny bit of overhead.

That approach is a little inconsistent, as well. Remember how I handle OAM: I write to a buffer, which is then copied to real OAM during the next vblank. But I’m updating the sprite immediately. That means when Anise turns, the sprite updates on the very next frame, but the movement isn’t visible until the frame after that. Whoops.

So, a buffer! I could make this into a more general mechanism later, but for now I only care about fixing Anise. I can revisit this when I have, uh, a second sprite.

1
2
3
4
; in ram somewhere

anise_sprites_address:
    dw

Now, Anise is composed of three objects, which is six chars, which is 96 bytes. The fastest way to copy bytes by hand is something like this:

1
2
3
4
5
6
7
8
9
    ld hl, source
    ld de, destination
    ld c, 96
.loop:
    ld a, [hl+]
    ld [de], a
    inc de
    dec c
    jr nz, .loop

Each iteration of the loop copies 1 byte and takes 7 cycles. (It’s possible to shave a couple cycles off in some specific cases, and unrolling would save some time, but let’s stay general for now.) That’s 672 cycles, plus 10 for the setup, minus one on the final jr, for 681 total. But vblank only lasts 1140 cycles! That’s more than half the budget blown for updating a single entity. This can’t possibly work.

Enter a feature exclusive to the Game Boy Color: GDMA, or general DMA. This is similar to OAM DMA, except that it can copy (nearly) anything to anywhere. Also (unlike OAM DMA), the CPU pauses while the copy is taking place, so there’s no need to carefully time a busy loop. It’s configured by writing to five control registers (which takes 5 cycles each), and then it copies two bytes per cycle, for a total of 73 cycles. That’s 9.3 times faster. Seems worth a try.

(Note that I’m not using double-speed CPU mode yet, as an incentive to not blow my CPU budget early on. Turning that on would halve the time taken by the manual loop, but wouldn’t affect GDMA.)

GDMA has a couple restrictions: most notably, it can only copy multiples of 16 bytes, and only to/from addresses that are aligned to 16 bytes. But each char is 16 bytes, so that works out just fine.

The five GDMA registers are, alas, simply named 1 through 5. The first two are the source address; the next two are the destination address; the last is the amount to copy. Or, well, it’s the amount to copy, divided by 16, minus 1. (The high bit is reserved for turning on a different kind of DMA that operates a bit at a time during hblanks.) Writing to the last register triggers the copy.

Plugging in this buffer is easy enough, then:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
    ; Update Anise's current sprite.  Use DMA here because...
    ; well, geez, it's too slow otherwise.
    ld hl, anise_sprites_address
    ld a, [hl+]
    ld [rHDMA1], a
    ld a, [hl]
    ld [rHDMA2], a
    ; I want to write to $8000 which is where Anise's sprite is
    ; hardcoded to live, and the top three bits are ignored so
    ; that the destination is always in VRAM, so $0000 works too
    ld a, HIGH($0000)
    ld [rHDMA3], a
    ld a, LOW($0000)
    ld [rHDMA4], a
    ; And copy!
    ld a, (32 * 3) / 16 - 1
    ld [rHDMA5], a

Finally, instead of actually overwriting Anise’s sprite, I write the address of the new sprite into the buffer:

1
2
3
4
5
    ; Store the new sprite address, to be updated during vblank
    ld a, h
    ld [anise_sprites_address], a
    ld a, l
    ld [anise_sprites_address + 1], a

And done! Now I can walk around just fine. It looks basically like the screenshot from the previous section, so I don’t think you need a new one.

Note that this copy will always happen, since there’s no condition for skipping it when there’s nothing to do. That’s fine for now; later I’ll turn this into a list, and after copying everything I’ll simply clear the list.

Crisis averted, or at least deferred until later. Back to the dialogue!

Interlude: A font

Writing out the glyphs by hand is not going to cut it. It was fairly annoying for two letters, let alone an entire alphabet.

Nothing about this part was especially interesting. I used LÖVE’s font format, which puts all glyphs in a single horizontal strip. The color of the top-left pixel is used as a sentinel; any pixel in the top row that’s the same color indicates the start of a new glyph.

(I note that LÖVE actually recommends against using this format, but the alternatives are more complicated and require platform-specific software — whereas I can slop this format together in any image editor without much trouble.)

I then turned this into Game Boy tiles much the same way as with the sprite loader, except with the extra logic to split on the sentinel pixels and pad each glyph to eight pixels wide. I won’t reproduce the whole script here, but it’s on GitHub if you want to see it.

The font itself is, well, a font? I initially tried to give it a little personality, but that made some of the characters weirdly wide and was a bit hard to read, so I revisited it and ended up with this:

Pixel font covering all of ASCII

I like it, at least! The characters all have shadows built right in, and you can see at the end that I was starting to play with some non-ASCII characters. Because I can do that!

Third pass

One major obstacle remains: I can only have one line of text right now, when there’s plenty of space for two.

The obvious first thing I need to do is alter the dialogue box’s char map. It currently has a whole char’s worth of padding on every side. What a waste. I want this instead:

1
2
3
4
5
6
7
8
9
+--+--+--+--+--+--+--+--+--+--+--+--+---+
|80|82|84|86|88|8a|8c|8e|90|92|94|96|...|
+--+--+--+--+--+--+--+--+--+--+--+--+---+
|81|83|85|87|89|8b|8d|8f|91|93|95|97|...|
+--+--+--+--+--+--+--+--+--+--+--+--+---+
|a8|aa|ac|ae|b0|b2|b4|b6|b8|ba|bc|be|...|
+--+--+--+--+--+--+--+--+--+--+--+--+---+
|a9|ab|ad|af|b1|b3|b5|b7|b9|bb|bd|bf|...|
+--+--+--+--+--+--+--+--+--+--+--+--+---+

The second row begins with char $a8 because that’s $80 + 40.

Obviously I’ll need to change the setup code to make the above pattern. But while I’m in here… remember, the setup code is the only remaining place that disables the LCD to do its work. Can I do everything within vblank instead?

I’m actually not sure, but there’s an easy way to reduce the CPU cost. Instead of setting up the whole dialogue box at once, I can do it one row at a time, starting from the bottom. That will cut the vblank pressure by a factor of four, and it’ll create a cool slide-up effect when the dialogue box opens!

Let’s give it a try. I’ll move the real code into a function, since it’ll run multiple times now. I’ll also introduce a few constants, since I’m getting tired of all the magic numbers everywhere.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
SCREEN_WIDTH_TILES EQU 20
CANVAS_WIDTH_TILES EQU 32
SCREEN_HEIGHT_TILES EQU 18
CANVAS_HEIGHT_TILES EQU 32
BYTES_PER_TILE EQU 16
TEXT_START_TILE_1 EQU 128
TEXT_START_TILE_2 EQU TEXT_START_TILE_1 + SCREEN_WIDTH_TILES * 2

; Fill a row in the tilemap in a way that's helpful to dialogue.
; hl: where to start filling
; b: tile to start with
fill_tilemap_row:
    ; Populate bank 0, the tile proper
    xor a
    ldh [rVBK], a

    ld c, SCREEN_WIDTH_TILES
    ld a, b
.loop0:
    ld [hl+], a
    ; Each successive tile in a row increases by 2!
    add a, 2
    dec c
    jr nz, .loop0

    ; Populate bank 1, the bank and palette
    ld a, 1
    ldh [rVBK], a
    ld a, %00001111  ; bank 1, palette 7
    ld c, SCREEN_WIDTH_TILES
    dec hl
.loop1:
    ld [hl-], a
    dec c
    jr nz, .loop1

    ret

Now replace the setup code with four calls to this function, waiting for vblank between successive calls.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
    ; Row 4
    ld hl, $9800 + CANVAS_WIDTH_TILES * (SCREEN_HEIGHT_TILES - 1)
    ld b, TEXT_START_TILE_2 + 1
    call fill_tilemap_row

    ; Row 3
    call wait_for_vblank
    ld hl, $9800 + CANVAS_WIDTH_TILES * (SCREEN_HEIGHT_TILES - 2)
    ld b, TEXT_START_TILE_2
    call fill_tilemap_row

    ; Row 2
    call wait_for_vblank
    ld hl, $9800 + CANVAS_WIDTH_TILES * (SCREEN_HEIGHT_TILES - 3)
    ld b, TEXT_START_TILE_1 + 1
    call fill_tilemap_row

    ; Row 1
    call wait_for_vblank
    ld hl, $9800 + CANVAS_WIDTH_TILES * (SCREEN_HEIGHT_TILES - 4)
    ld b, TEXT_START_TILE_1
    call fill_tilemap_row

Cool. I have a full font now, too, so I might as well try it out with some more interesting text.

1
2
3
SECTION "Font", ROMX
text:
    db "The quick brown fox jumps over the     lazy dog's back.  AOOWWRRR!!!!", 0

Now I just need to— oh, hang on.

Animation of the text box sliding up and scrolling out the text

Hey, it already works! Magic.

(I did also change the initial value for the x-offset to 4 rather than 0, so the text doesn’t start against the left edge of the screen.)

Well. Not really. The code I wrote doesn’t actually know when to stop writing, so it continues off the end of the first line and onto the second. You may notice the conspicuous number of extra spaces in the new text.

Still, it looks right, and this was a lot of effort already, and it’s not actually plugged into anything yet, so I called this a success and shelved it for now. Quit while you’re ahead, right?

Future work

Obviously this is still a bit rough.

That thing where the player can walk on top of the textbox is a bit of a problem, since the same thing happens if the textbox opens while the player is near the bottom of the screen. There are a couple solutions to this, and they’ll really depend on how I end up deciding to display the box.

I actually wanted the glyphs to be drawn a little lower than normal on the top line, to add half a char or so of padding around them, but I tried it and got a buffer overrun that I didn’t feel like investigating. That’s an obvious thing to fix next time I touch this code.

What about word wrapping? I’ve written about that before and clearly have strong opinions about it, but I really don’t want to do dynamic word wrapping with a variable-width font on a Game Boy. Instead, I’ll probably store dialogue in some other format and use another converter script to do the word-wrapping ahead of time. That’ll also save me from writing large amounts of dialogue in, um, assembly. And if/when I want any fancy-pants special effects within dialogue, I can describe them with a human-readable format and then convert that to more assembly-friendly bytecode instructions.

The dialogue box still doesn’t go away, partly because it draws right on top of the map, and I don’t have any easy way to repair the map right now. I’ll probably switch to one of those other mechanisms for showing the box later that won’t require clobbering the map, and then this problem will pretty much solve itself.

What about menus? Those will either have to go inside the dialogue box (which means the question being asked isn’t visible, oof), or they’ll have to go in a smaller box above it like in Pokémon. But the latter solution means I can’t use the window or display trickery — both of those only work reliably for horizontal splits. I’m not quite sure how to handle this, yet.

And then, what of portraits? Most games get away without them by having a silent protagonist, which makes it obvious who’s talking. But Anise is anything but silent, so I need a stronger indicator. I obviously can’t overlay a big transparent portrait on the background, like I do in my LÖVE games. I think I can reseve space for them in the status bar, which will go underneath the dialogue box. I’ll have to see how it works out. Maybe I could also use a different text color for every speaker?

After all that, I can start worrying about other frills like colored text and pauses and whatever. Phew.

To be continued

That brings us up to commit a173db, which is slightly beyond the second release (which includes a one-line textbox)! Also that was three months ago oh dear. I think I’ll be putting out a new release soon, stay tuned!

Next time: collision detection! I am doomed.

Cheezball Rising: Opening a dialogue

Post Syndicated from Eevee original https://eev.ee/blog/2018/09/08/cheezball-rising-opening-a-dialogue/

This is a series about Star Anise Chronicles: Cheezball Rising, an expansive adventure game about my cat for the Game Boy Color. Follow along as I struggle to make something with this bleeding-edge console!

GitHub has intermittent prebuilt ROMs, or you can get them a week early on Patreon if you pledge $4. More details in the README!


In this issue, I draw some text!

Previously: I get a Game Boy to meow.
Next: collision detection, ohh nooo

Recap

The previous episode was a diversion (and left an open problem that I only solved after writing it), so the actual state of the game is unchanged.

Star Anise walking around a moon environment in-game, animated in all four directions

Where should I actually go from here? Collision detection is an obvious place, but that’s hard. Let’s start with something a little easier: displaying scrolling dialogue text. This is likely to be a dialogue-heavy game, so I might as well get started on that now.

Planning

On any other platform, I’d dive right into it: draw a box on the screen somewhere, fill it with text.

On the Game Boy, it’s not quite that simple. I can’t just write text to the screen; I can only place tiles and sprites.

Let’s look at how, say, Pokémon Yellow handles its menu.

Pokémon Yellow with several levels of menu open

This looks — feels — like it’s being drawn on top of the map, and that sub-menus open on top of other menus. But it’s all an illusion! There’s no “on top” here. This is a completely flat image made up of tiles, like anything else.

The same screenshot, scaled up, with a grid showing the edges of tiles

This is why Pokémon has such a conspicuously blocky font: all the glyphs are drawn to fit in a single 8×8 char, so “drawing” text is as simple as mapping letters to char indexes and drawing them onto the background. The map and the menu are all on the same layer, and the game simply redraws whatever was underneath when you close something. Part of the illusion is that the game is clever enough to hide any sprites that would overlap the menu — because sprites would draw on top! (The Game Boy Color has some twiddles for controlling this layering, but Yellow was originally designed for the monochrome Game Boy.)

A critical reason that this actually works is that in Pokémon, the camera is always aligned to the grid. It scrolls smoothly while you’re walking, but you can’t actually open the menu (or pick up an item, or talk to someone, or do anything else that might show text) until you’ve stopped moving. If you could, the menu would be misaligned, because it’s part of the same grid as the map!

This poses a slight problem for my game. Star Anise isn’t locked to the grid like the Pokémon protagonist is, and unlike Link’s Awakening, I do want to have areas larger than the screen that can scroll around freely.

I know offhand that there are a couple ways to do this. One is the window, an optional extra opaque layer that draws on top of the background, with its top-left corner anchored to any point on the screen. Another is to change some display registers in the middle of the screen redrawing. The Oracle games combine both features to have a status bar at the top of the screen but a scrolling map underneath.

But I don’t want to worry about any of this right now, before I even have text drawing. I know it’s possible, so I’ll deal with it later. For now, drawing directly onto the background is good enough.

Font decisions

Let’s get back to the font itself. I’m not in love with the 8×8 aesthetic; what are my other options? I do like the text in Oracle of Ages, so let’s have a look at that:

Oracle of Ages, also scaled up with a grid, showing its taller text

Ah, this is the same approach again, except that letters are now allowed to peek up into the char above. So these are 8×16, but the letters all occupy a box that’s more like 6×9, offering much more familiar proportions. Oracle of Ages is designed for the Game Boy Color, which has twice as much char storage space, so it makes sense that they’d take advantage of it for text like this.

It’s not bad, but the space it affords is still fairly… limited. Only 16 letters will fit in a line, just as with Pokémon, and that means a lot of carefully wording things to be short and use mostly short words as well. That’s not gonna cut it for the amount of dialogue I expect to have.

What other options do I have? It seems like I’m limited to multiples of 8 here, surely. (The answer may be obvious to some of you, but shh, don’t read ahead.)

The answer lies in the very last game released for the Game Boy Color: Harry Potter and the Chamber of Secrets. Whatever deep secrets were learned during the Game Boy’s lifetime will surely be encapsulated within this, er, movie tie-in game.

Harry Potter and the Chamber of Secrets, also scaled up with a grid, showing its text isn't fixed to the grid

Hot damn. That is a ton of text in a relatively small amount of space! And it doesn’t fit the grid! How did they do that?

The answer is… exactly how you’d think!

Tile display for the above screenshot, showing that the text is simply written across consecutive tiles

With a fixed-width font like in Pokémon and Zelda games, the entire character set is stored in VRAM, and text is drawn by drawing a string of characters. With a variable-width font like in Harry Potter, a block of VRAM is reserved for text, and text is drawn into those chars, in software. Essentially, some chars are used like a canvas and have text rendered to them on the fly. The contents of the background layer might look like this in the two cases:

Illustration of fixed width versus variable width text

Some pros of this approach:

  • Since the number of chars required is constant and the font is never loaded directly into char memory, the font can have arbitrarily many glyphs in it. Multiple fonts could be used at the same time, even. (Of course, if you have more than 256 glyphs, you’ll have to come up with a multi-byte encoding for actually storing the text…)

  • A lot more text can fit in one line while still remaining readable.

  • It has the potential to look extremely cool and maybe even vaguely technically impressive.

And, cons:

  • It’s definitely more complicated! But I only have to write the code once, and since the game won’t be doing anything but drawing dialogue while the box is up, I don’t think I’ll be in danger of blowing my CPU budget.

  • Colored text becomes a bit trickier. But still possible, so, we can worry about that later.

Well, I’m sold. Let’s give it a shot.

First pass

Well, I want to do something on a button press, so, let’s do that.

A lot of games (older ones especially) have bugs from switching “modes” in the same frame that something else happens. I don’t entirely understand why that’s so common and should probably ask some speedrunners, but I should be fine if I do mode-switching first thing in the frame, and then start over a new frame when switching back to “world” mode. Right? Sure.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
    ; ... button reading code in main loop ...
    bit BUTTON_A, a
    jp nz, .do_show_dialogue

    ; ... main loop ...

    ; Loop again when done
    jp vblank_loop

.do_show_dialogue:
    call show_dialogue
    jp vblank_loop

The extra level of indirection added by .do_show_dialogue is just so the dialogue code itself isn’t responsible for knowing where the main loop point is; it can just ret.

Now to actually do something. This is a first pass, so I want to do as little as possible. I’ll definitely need a palette for drawing the text — and here I’m cutting into my 8-palette budget again, which I don’t love, but I can figure that out later. (Maybe with some shenanigans involving changing the palettes mid-redraw, even.)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
PALETTE_TEXT:
    ; Black background, white text...  then gray shadow, maybe?
    dcolor $000000
    dcolor $ffffff
    dcolor $999999
    dcolor $666666

show_dialogue:
    ; Have to disable the LCD to do video work.  Later I can do
    ; a less jarring transition
    DisableLCD

    ; Copy the palette into slot 7 for now
    ld a, %10111000
    ld [rBCPS], a
    ld hl, PALETTE_TEXT
    REPT 8
    ld a, [hl+]
    ld [rBCPD], a
    ENDR

I also know ahead of time what chars will need to go where on the screen, so I can fill them in now.

Note that I really ought to blank them all out, especially since they may still contain text from some previous dialogue, but I don’t do that yet.

An obvious question is: which tiles? I think I said before that with 512 chars available, and ¾ of those still being enough to cover the entire screen in unique chars, I’m okay with dedicating a quarter of my space to UI stuff, including text. To keep that stuff “out of the way”, I’ll put them at the “end” — bank 1, starting from $80.

I’m thinking of having characters be about the same proportions as in the Oracle games. Those games use 5 rows of tiles, like this:

1
2
3
4
5
top of line 1
bottom of line 1
top of line 2
bottom of line 2
blank

Since the font is aligned to the bottom and only peeks a little bit into the top char, the very top row is mostly blank, and that serves as a top margin. The bottom row is explicitly blank for a bottom margin that’s nearly the same size. The space at the top of line 2 then works as line spacing.

I’m not fixed to the grid, so I can control line spacing a little more explicitly. But I’ll get to that later and do something really simple for now, where $ff is a blank tile:

1
2
3
4
5
6
7
8
9
+--+--+--+--+--+--+--+--+--+--+--+--+--+---+
|ff|ff|ff|ff|ff|ff|ff|ff|ff|ff|ff|ff|ff|...|
+--+--+--+--+--+--+--+--+--+--+--+--+--+---+
|ff|80|82|84|86|88|8a|8c|8e|90|92|94|96|...|
+--+--+--+--+--+--+--+--+--+--+--+--+--+---+
|ff|81|83|85|87|89|8b|8d|8f|91|93|95|97|...|
+--+--+--+--+--+--+--+--+--+--+--+--+--+---+
|ff|ff|ff|ff|ff|ff|ff|ff|ff|ff|ff|ff|ff|...|
+--+--+--+--+--+--+--+--+--+--+--+--+--+---+

This gives me a canvas for drawing a single line of text. The staggering means that the first letter will draw to adjacent chars $80 and $81, rather than distant cousins like $80 and $a0.

You may notice that the below code updates chars across the entire width of the grid, not merely the screen. There’s not really any good reason for that.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
    ; Fill text rows with tiles (blank border, custom tiles)
    ; The screen has 144/8 = 18 rows, so skip the first 14 rows
    ld hl, $9800 + 32 * 14
    ; Top row, all tile 255
    ld a, 255
    ld c, 32
.loop1:
    ld [hl+], a
    dec c
    jr nz, .loop1

    ; Text row 1: 255 on the edges, then middle goes 128, 130, ...
    ld a, 255
    ld [hl+], a
    ld a, 128
    ld c, 30
.loop2:
    ld [hl+], a
    add a, 2
    dec c
    jr nz, .loop2
    ld a, 255
    ld [hl+], a

    ; Text row 2: same as above, but middle is 129, 131, ...
    ld a, 255
    ld [hl+], a
    ld a, 129
    ld c, 30
.loop3:
    ld [hl+], a
    add a, 2
    dec c
    jr nz, .loop3
    ld a, 255
    ld [hl+], a

    ; Bottom row, all tile 255
    ld a, 255
    ld c, 32
.loop4:
    ld [hl+], a
    dec c
    jr nz, .loop4

Now I need to repeat all of that, but in bank 1, to specify the char bank (1) and palette (7) for the corresponding tiles. Those are the same for the entire dialogue box, though, so this part is easier.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
    ; Switch to VRAM bank 1
    ld a, 1
    ldh [rVBK], a

    ld a, %00001111  ; bank 1, palette 7
    ld hl, $9800 + 32 * 14
    ld c, 32 * 4  ; 4 rows
.loop5:
    ld [hl+], a
    dec c
    jr nz, .loop5

    EnableLCD

Time to get some real work done. Which raises the question: how do I actually do this?

If you recall, each 8-pixel row of a char is stored in two bytes. The two-bit palette index for each pixel is split across the corresponding bit in each byte. If the leftmost pixel is palette index 01, then bit 7 in the first byte will be 0, and bit 7 in the second byte will be 1.

Now, a blank char is all zeroes. To write a (left-aligned) glyph into a blank char, all I need to do is… well, I could overwrite it, but I could just as well OR it. To write a second glyph into the unused space, all I need to do is shift it right by the width of the space used so far, and OR it on top. The unusual split layout of the palette data is actually handy here, because it means the size of the shift matches the number of pixels, and I don’t have to worry about overflow.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
0 0 0 0 0 0 0 0  <- blank glyph

1 1 1 1 0 0 0 0  <- some byte from the first glyph
↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
1 1 1 1 0 0 0 0  <- ORed together to display first character

          1 1 1 1 0 0 0 0  <- some byte from the second glyph,
                              shifted by 4 (plus a kerning pixel)
↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
1 1 1 1 0 1 1 1  <- ORed together to display first two characters

The obvious question is, well, what happens to the bits from the second character that didn’t fit? I’ll worry about that a bit later.

Oh, and finally, I’ll need a font, plus some text to display. This is still just a proof of concept, so I’ll add in a couple glyphs by hand.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
; somewhere in ROM
font:
; A
    ; First byte indicates the width of the glyph, which I need
    ; to know because the width varies!
    db 6
    dw `00000000
    dw `00000000
    dw `01110000
    dw `10001000
    dw `10001000
    dw `10001000
    dw `11111000
    dw `10001000
    dw `10001000
    dw `10001000
    dw `10001000
    dw `00000000
    dw `00000000
    dw `00000000
    dw `00000000
    dw `00000000
; B
    db 6
    dw `00000000
    dw `00000000
    dw `11110000
    dw `10001000
    dw `10001000
    dw `10001000
    dw `11110000
    dw `10001000
    dw `10001000
    dw `10001000
    dw `11110000
    dw `00000000
    dw `00000000
    dw `00000000
    dw `00000000
    dw `00000000

text:
    ; Shakespeare it ain't.
    ; Need to end with a NUL here so I know where the text
    ; ends.  This isn't C, there's no automatic termination!
    db "ABABAAA", 0

And here we go!

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
    ; ----------------------------------------------------------
    ; Setup done!  Real work begins here
    ; b: x-offset within current tile
    ; de: text cursor + current character tiles
    ; hl: current VRAM tile being drawn into
    ld b, 0
    ld de, text
    ld hl, $8800

    ; This loop waits for the next vblank, then draws a letter.
    ; Text thus displays at ~60 characters per second.
.next_letter:
    ; This is probably way more LCD disabling than is strictly
    ; necessary, but I don't want to worry about it yet
    EnableLCD
    call wait_for_vblank
    DisableLCD

    ld a, [de]                  ; get current character
    and a                       ; if NUL, we're done!
    jr z, .done
    inc de                      ; otherwise, increment

    ; Get the glyph from the font, which means computing
    ; font + 33 * a.
    ; A little register juggling.  hl points to the current
    ; char in VRAM being drawn to, but I can only do a 16-bit
    ; add into hl.  de I don't need until the next loop,
    ; since I already read from it.  So I'm going to push de
    ; AND hl, compute the glyph address in hl, put it in de,
    ; then restore hl.
    push de
    push hl
    ; The text is written in ASCII, but the glyphs start at 0
    sub a, 65
    ld hl, font
    ld de, 33                   ; 1 width byte + 16 * 2 tiles
    ; This could probably be faster with long multiplication
    and a
.letter_stride:
    jr z, .skip_letter_stride
    add hl, de
    dec a
    jr .letter_stride
.skip_letter_stride:
    ; Move the glyph address into de, and restore hl
    ld d, h
    ld e, l
    pop hl

    ; Read the first byte, which is the character width.  This
    ; overwrites the character, but I have the glyph address,
    ; so I don't need it any more
    ld a, [de]
    inc de

    ; Copy into current chars
    ; Part 1: Copy the left part into the current chars
    push af                     ; stash width
    ; A glyph is two chars or 32 bytes, so row_copy 32 times
    ld c, 32
    ; b is the next x position we're free to write to.
    ; Incrementing it here makes the inner loop simpler, since
    ; it can't be zero.  But it also means two jumps per loop,
    ; so, ultimately this was a pretty silly idea.
    inc b
.row_copy:
    ld a, [de]                  ; read next row of character

    ; Shift right by b places with an inner loop
    push bc                     ; preserve b while shifting
    dec b
.shift:                         ; shift right by b bits
    jr z, .done_shift
    srl a
    dec b
    jr .shift
.done_shift:
    pop bc

    ; Write the updated byte to VRAM
    or a, [hl]                  ; OR with current tile
    ld [hl+], a
    inc de
    dec c
    jr nz, .row_copy
    pop af                      ; restore width

    ; Part 2: Copy whatever's left into the next char
    ; TODO  :)

    ; Cleanup for next iteration
    ; Undo the b increment from way above
    dec b
    ; It's possible I overflowed into the next column, in which
    ; case I want to leave hl where it is: pointing at the next
    ; column.  Otherwise, I need to back it up to where it was.
    ; Of course, I also need to update b, the x offset.
    add a, b                    ; a <- new x offset
    ; If the new x offset is 8 or more, that's actually the next
    ; column
    cp a, 8
    jr nc, .wrap_to_next_tile
    ld bc, -32                  ; a < 8: back hl up
    add hl, bc
    jr .done_wrap
.wrap_to_next_tile:
    sub a, 8                    ; a >= 8: subtract tile width
    ld b, a
.done_wrap:
    ; Either way, store the new x offset into b
    ld b, a

    ; And loop!
    pop de                      ; pop text pointer
    jr .next_letter

.done:
    ; Undo any goofy stuff I did, and get outta here
    EnableLCD
    ; Remember to reset bank to 0!
    xor a
    ldh [rVBK], a
    ret

Phew! That was a lot, but hopefully it wasn’t too bad. I hit a few minor stumbling blocks, but as I recall, most of them were of the “I get the conditions backwards every single time I use cp augh” flavor. (In fact, if you look at the actual commit the above is based on, you may notice that I had the condition at the very end mixed up! It’s a miracle it managed to print part of the second letter at all.)

There are a lot of caveats in this first pass, including that there’s nothing to erase the dialogue box and reshow the map underneath it. (But I might end up using the window for this anyway, so there’s no need for that.)

As a proof of concept, though, it’s a great start!

Screenshot of Anise, with a black dialogue box that says: A|

That’s the letter A, followed by the first two pixel of the letter B. I didn’t implement the part where letters spill into the next column, yet.

Guess I’d better do that!

Second pass

One of the big problems with the first pass was that I had to turn the screen off to do the actual work safely. Shifting a bunch of bytes by some amount is a little slow, since I can only shift one bit at a time and have to do it within a loop, and vblank only lasts for about 6.5% of the entire duration of the frame.

SECTION “Text buffer”, WRAM0[$C200]
text_buffer:
; Text is up to 8×16 but may span two columns, so carve out
; enough space for four tiles
ds $40

SECTION “Text rendering”, ROM0
PALETTE_TEXT:
dcolor $000000
dcolor $ffffff
dcolor $999999
dcolor $666666

show_dialogue:
; TODO blank out the second half of bank 1 before all this, maybe on the fly to average out the cpu time

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
; TODO get rid of this with a slide-up effect
DisableLCD

; Set up palette
ld a, %10111000
ld [rBCPS], a
ld hl, PALETTE_TEXT
REPT 8
ld a, [hl+]
ld [rBCPD], a
ENDR

; Fill text rows with tiles (blank border, custom tiles)
ld hl, $9800 + 32 * 14
; Top row, all tile 255
ld a, 255
ld c, 32

.loop1:
ld [hl+], a
dec c
jr nz, .loop1
; Text row 1: 255 on the edges, then middle goes 128, 130, …
ld a, 255
ld [hl+], a
ld a, 128
ld c, 30
.loop2:
ld [hl+], a
add a, 2
dec c
jr nz, .loop2
ld a, 255
ld [hl+], a
; Text row 2: same as above, but middle is 129, 131, …
ld a, 255
ld [hl+], a
ld a, 129
ld c, 30
.loop3:
ld [hl+], a
add a, 2
dec c
jr nz, .loop3
ld a, 255
ld [hl+], a
; Bottom row, all tile 255
ld a, 255
ld c, 32
.loop4:
ld [hl+], a
dec c
jr nz, .loop4

1
2
3
4
5
6
7
; Repeat all of the above, but in bank 1, which specifies the character bank and palette.  Luckily, that's the same for everyone.
ld a, 1
ldh [rVBK], a
ld a, %00001111  ; bank 1, palette 7
ld hl, $9800 + 32 * 14
; Top row, all tile 255
ld c, 32 * 4  ; 4 rows

.loop5:
ld [hl+], a
dec c
jr nz, .loop5

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
EnableLCD

; Zero out the tile buffer
xor a
ld hl, text_buffer
ld c, $40
call fill

; ----------------------------------------------------------
; Setup done!  Real work begins here
; b: x-offset within current tile
; de: text cursor + current character tiles
; hl: current VRAM tile being drawn into + buffer pointer
ld b, 0
ld de, text
ld hl, $8800

; The basic problem here is to shift a byte and split it
; across two other bytes, like so:
;      yyyyy YYY
;   xxx00000 00000000
;           ↓
;   xxxyyyyy YYY00000
; To do this, we rotate the byte, mask the low bits, OR them
; with the first byte, restore it, mask the high bits, and
; then store that directly as the second byte (which should
; be all zeroes anyway).

.next_letter:
ld a, [de] ; get current character
and a ; if NUL, we’re done!
jp z, .done
inc de ; otherwise, increment

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
; Get the font character
push de                     ; from here, de is tiles
; Alas, I can only add to hl, so I need to compute the font
; character address in hl and /then/ put it in de.  But I
; already pushed de, so I can use that as scratch space.
push hl
sub a, 65   ; TODO temporary
ld hl, font
ld de, 33                   ; 1 width byte + 16 * 2 tiles
; TODO can we speed striding up with long mult?
and a

.letter_stride:
jr z, .skip_letter_stride
add hl, de
dec a
jr .letter_stride
.skip_letter_stride:
ld d, h ; move char tile addr to de
ld e, l

1
2
3
4
5
6
7
8
ld a, [de]                  ; read width
inc de

; Copy into current tiles
push af                     ; stash width
ld c, 32                    ; 32 bytes per row
ld hl, text_buffer
inc b   ; FIXME? this makes the loop simpler since i only test after the dec, but it also is the 1px kerning between characters...

.row_copy:
ld a, [de] ; read next row of character
; Rotate right by b – 1 pixels
push bc ; preserve b while shifting
ld c, $ff ; create a mask
dec b
jr z, .skip_rotate
.rotate:
rrca
srl c
dec b
jr nz, .rotate
.skip_rotate:
push af
and a, c ; mask right pixels
; Draw to left half of text buffer
or a, [hl] ; OR with current tile
ld [hl+], a
; Write the remaining bits to right half
ld a, c ; put mask in a…
cpl ; …to invert it
ld c, a ; then put it back
pop af ; restore unmasked pixels
and a, c ; mask left pixels
ld [hl+], a ; and store them!
; Loop and cleanup
inc de ; next row of character
pop bc ; restore counter!
dec c
jr nz, .row_copy
pop af ; restore width

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
; Draw the buffered tiles to vram
; The text buffer is treated like it's 16 pixels wide, but
; VRAM is of course only 8 pixels wide, so we need to do
; this in two iterations: the left two tiles, then the right
; TODO explain this with a fucking diagram because i feel
; like i'm wrong about it anyway
pop hl                      ; restore hl (VRAM)
push af                     ; stash width, again
call wait_for_vblank        ; always wait before drawing
push bc
push de
; Draw the left two tiles
ld c, $20
ld de, text_buffer

.draw_left:
ld a, [de]
inc de
inc de
ld [hl+], a
dec c
jr nz, .draw_left
; Draw the right two tiles
ld c, $20
ld de, text_buffer + 1
.draw_right:
ld a, [de]
inc de
inc de
ld [hl+], a
dec c
jr nz, .draw_right
pop de
pop bc
pop af ; restore width, again

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
; Increment the pixel offset and deal with overflow
; TODO it's possible we're at 9 pixels wide, thanks to the
; kerning pixel, uh oh.  but that pixel would be empty,
; right?  wait, no, it comes /before/...  well fuck
; TODO actually that might make something weird happen due
; to the inc b above, maybe...?
add a, b                    ; a <- new x offset
ld bc, -32                  ; move the VRAM pointer back...
add hl, bc                  ; ...to the start of the tile
cp a, 8
jr nc, .wrap_to_next_tile
; The new offset is less than 8, so this character didn't
; draw into the next tile.  Move the VRAM pointer back
; another two tiles, to the column we started in
add hl, bc
jr .done_wrap

.wrap_to_next_tile:
; The new offset is 8 or more, so this character drew into
; the next tile. Subtract 8, but also shift the text buffer
; by copying all the “right” tiles over the “left” tiles
sub a, 8 ; a >= 8: subtract tile width
push hl
push af
ld hl, text_buffer + $40 – 1
ld c, $20
.shift_buffer:
ld a, [hl-]
ld [hl-], a
dec c
jr nz, .shift_buffer
pop af
pop hl
.done_wrap:
ld b, a ; either way, store into b

1
2
3
; Loop
pop de                      ; pop text pointer
jp .next_letter

.done:
EnableLCD ; TODO get rid of me with a buffer
; Remember to reset bank to 0!
xor a
ldh [rVBK], a ret

wait_for_vblank:
xor a ; clear the vblank flag
ld [vblank_flag], a
.vblank_loop:
halt ; wait for interrupt
ld a, [vblank_flag] ; was it a vblank interrupt?
and a
jr z, .vblank_loop ; if not, keep waiting ret

  • future ideas: how will this work with a status bar, how do i do portraits, how do i hide sprites behind this, how do i handle the map not being aligned (contrast with pokemon which draws the entire menu on the background)

lingering problems
– note on word wrapping

  • alignment, window
  • prompts will probably have to go inside the text box? hmm. that’s tricky.
  • portraits!

content/2016-10-20-word-wrapping-dialogue.markdown
– the dialogue box does not actually go away. but i think the window will solve this

To be continued

This work doesn’t correspond to a commit at all; it exists only as a local stash. I’ll clean it up later, once I figure out what to actually do with it.

Next time: dialogue! With moderately less suffering along the way!

Cheezball Rising: Resounding failure

Post Syndicated from Eevee original https://eev.ee/blog/2018/09/06/cheezball-rising-resounding-failure/

This is a series about Star Anise Chronicles: Cheezball Rising, an expansive adventure game about my cat for the Game Boy Color. Follow along as I struggle to make something with this bleeding-edge console!

GitHub has intermittent prebuilt ROMs, or you can get them a week early on Patreon if you pledge $4. More details in the README!


In this issue, I cannot get a goddamn Game Boy to meow at me.

Previously: maps and sprites.
Next: text!

Recap

With the power of Aseprite, Tiled, and some Python I slopped together, the game has evolved beyond Test Art and into Regular Art.

Star Anise walking around a moon environment in-game, animated in all four directions

I’ve got so much work to do on this, so it’s time to prioritize. What is absolutely crucial to this game?

The answer, of course, is to make Anise meow. Specifically, to make him AOOOWR.

Brief audio primer

What we perceive as sound is the vibration of our eardrums, caused by vibration of the air against them. Eardrums can only move along a single axis (in or out), so no matter what chaotic things the air is doing, what we hear at a given instant is flattened down to a single scalar number: how far the eardrum has displaced from its normal position.

(There’s also a bunch of stuff about tiny hairs in the back of your ear, but, close enough. Also it’s really two numbers since you have two ears, but stereo channels tend to be handled separately.)

Digital audio is nothing more than a sequence of those numbers. Of course, we can’t record the displacement at every single instant, because there are infinitely many instants; instead, we take measurements (samples) at regular intervals. The interval is called the sample rate, is usually a very small fraction of a second, and is generally measured in Hertz/Hz (which just means “per second”). A very common sample rate is 44100 Hz, which means a measurement was taken every 0.0000227 seconds.

I say “measurement” but the same idea applies for generating sounds, which is what the Game Boy does. Want to make a square wave? Just generate a block of all the same positive sample, then another block of all the same negative sample, and alternate back and forth. That’s why it’s depicted as a square — that’s the graph of how the samples vary over time.

Okay! I hope that was enough because it’s like 80% of everything I know about audio. Let’s get to the Game Boy.

Game Boy audio

The Game Boy contains, within its mysterious depths, a teeny tiny synthesizer. It offers a vast array of four whole channels (instruments) to choose from: a square wave, also a square wave, a wavetable, and white noise. They can each be controlled with a handful of registers, and will continually produce whatever tone they’re configured for. By changing their parameters at regular intervals, you can create a pleasing sequence of varying tones, which you humans call “music”.

Making music is, I’m sure, going to be an absolute nightmare. What music authoring tools am I possibly going to dig up that exactly conform to the Game Boy hardware? I can’t even begin to imagine what this pipeline might look like.

Luckily, that’s not what this post is about, because I chickened out and tried something way easier instead.

Before I set out into the wilderness myself, I did want to get an emulator to create any kind of noise at all, just to give myself a starting point. There are an awful lot of audio twiddles, so I dug up a Game Boy sound tutorial.

I became a little skeptical when the author admitted they didn’t know what a square wave was, but they did provide a brief snippet of code at the end that’s claimed to produce a sound:

1
2
3
4
5
6
7
8
9
NR52_REG = 0x80;
NR51_REG = 0x11;
NR50_REG = 0x77;

NR10_REG = 0x1E;
NR11_REG = 0x10;
NR12_REG = 0xF3;
NR13_REG = 0x00;
NR14_REG = 0x87;

That’s C, written for the much-maligned GBDK, which for some reason uses regular assignment to write to a specific address? It’s easy enough to translate to rgbasm:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
    ; Enable sound globally
    ld a, $80
    ldh [rAUDENA], a
    ; Enable channel 1 in stereo
    ld a, $11
    ldh [rAUDTERM], a
    ; Set volume
    ld a, $77
    ldh [rAUDVOL], a

    ; Configure channel 1.  See below
    ld a, $1e
    ldh [rAUD1SWEEP], a
    ld a, $10
    ldh [rAUD1LEN], a
    ld a, $f3
    ldh [rAUD1ENV], a
    ld a, $00
    ldh [rAUD1LOW], a
    ld a, $85
    ldh [rAUD1HIGH], a

It sounds like this.

Some explanation may be in order. This is a big ol’ mess and you could just as well read the wiki’s article on the sound controller, so feel free to skip ahead a bit.

First, the official names for all of the sound registers are terrible. They’re all named “NRxy” — “noise register” perhaps? — where x is the channel number (or 5 for master settings) and y is just whatever. Thankfully, hardware.inc provides some aliases that make a little more sense, and those are what I’ve used above.

The very first thing I have to do is set the high bit of AUDENA (NR52), which toggles sound on or off entirely. The sound system isn’t like the LCD, which I might turn off temporarily while doing a lot of graphics loading; when the high bit of AUDENA is off, all the other sound registers are wiped to zero and cannot be written until sound is enabled again.

The other important master registers are AUDVOL (NR50) and AUDTERM (NR51). Both of them are split into two identical nybbles, each controlling the left or right output channel. AUDVOL controls the master volume, from 0 to 7. (As I understand it, the high bit is used to enable audio output from extra synthesizer hardware on the cartridge, a feature I don’t believe any game ever actually used.) AUDTERM enables channels/instruments, one bit per channel. The above code turns on channel 1, the square wave, at max volume in stereo.

Then there’s just, you know, sound stuff.

AUD1HIGH (NR14) and AUD1LOW (NR13) are a bit of a clusterfuck, and one shared by all except the white noise channel. The high bit of AUD1HIGH is the “init” bit and triggers the sound to actually play (or restart), which is why it’s set last. The second highest bit, bit 6, controls timing: if it’s set, then the channel will only play for as long as a time given by AUD1LEN; if not, the channel will play indefinitely.

Finally, the interesting part: the lower three bits of AUD1HIGH and the entirety of AUD1LOW combine to make an 11-bit frequency. Or, rather, if those 11 bits are \(n\), then the frequency is \(\frac{131072}{2048-n}\). (Since their value appears in the denominator, they really express… inverse time, not frequency, but that’s neither here nor there.) The code above sets that 11-bit value to $500, for a frequency of 171 Hz, which in A440 is about an F3.

AUD1SWEEP (NR10) can automatically slide the frequency over time. It distinguishes channel 1 from channel 2, which is otherwise identical but doesn’t have sweep functionality. The lower three bits are the magnitude of each change; bit 3 is a sign bit (0 for up, 1 for down), and bits 6–4 are a time that control how often the frequency changes. (Setting the time to zero disables the sweep.) Given a magnitude of \(n\) and time \(t\), every \(\frac{t}{128}\) seconds, the frequency is multiplied by \(1 ± \frac{1}{2^n}\).

Note that when I say “frequency” here, I’m referring to the 11-bit “frequency” value, not the actual frequency in Hz. A “frequency” of $400 corresponds to 128 Hz, but halving it to $200 produces 85 Hz, a decrease of about a third. Doubling it is impossible, because $800 doesn’t fit in 11 bits. This setup seems, ah, interesting to make music with. Can’t wait!

The above code sets this register to $1e, so \(t = 1\), \(n = 6\), and the frequency is decreasing; thus every \(\frac{1}{128}\) seconds, the “frequency” drops by \(\frac{1}{64}\).

Next is AUD1LEN (NR11), so named because its lower six bits set how long the sound will play. Again we have inverse time: given a value \(t\) in the low six bits, the sound will play for \(\frac{64-t}{256}\) seconds. Here those six bits are &x#24;10 or 16, so the sound lasts for \(\frac{48}{256} = \frac{3}{16} = 0.1875\) seconds. Except… as mentioned above, this only applies if bit 6 of AUD1HIGH is set, which it isn’t, so this doesn’t apply at all and there’s no point in setting any of these bits. Hm.

The two high bits of AUD1LEN select the duty cycle, which is how long the square wave is high versus low. (A “normal” square wave thus has a duty of 50%.) Our value of 0 selects 12.5% high; the other values are 25% for 1, 50% for 2, or 75% for 3. I do wonder if the author of this code meant to use 50% duty and put the bit in the wrong place? If so, AUD1LEN should be $80, not $10.

Finally, AUD1ENV selects the volume envelope, which can increase or decrease over time. Curiously, the resolution is higher here than in AUDVOL — the entire high nybble is the value of the envelope. This value can be changed automatically over time in increments of 1: bit 3 controls the direction (0 to decrease, 1 to increase) and the low three bits control how often the value changes, counted in \(\frac{1}{64}\) seconds. For our value of $f3, the volume starts out at max and decreases every \(\frac{3}{64}\) seconds, so it’ll stop completely (or at least be muted?) after fifteen steps or \(\frac{45}{64} ≈ 0.7\) seconds.

And hey, that’s all more or less what I see if I record mGBA’s output in Audacity!

Waveform of the above sound

Boy! What a horrible slog. Don’t worry; that’s a good 75% of everything there is to know about the sound registers. The second square wave is exactly the same except it can’t do a frequency sweep. The white noise channel is similar, except that instead of frequency, it has a few knobs for controlling how the noise is generated. And the waveform channel is what the rest of this post is about—

Hang on!” I hear you cry. “That’s a mighty funny-looking ‘square’ wave.”

It sure is! The Game Boy has some mighty funny sound hardware. Don’t worry about it. I don’t have any explanation, anyway. I know the weird slope shapes are due to a high-pass filter capacitor that constantly degrades the signal gradually towards silence, but I don’t know why the waveform isn’t centered at zero. (Note that mGBA has a bug and currently generates audio inverted, which is hard to notice audibly but which means the above graph is upside-down.)

The thing I actually wanted to do

Right, back to the thing I actually wanted to do.

I have a sound. I want to play it on a Game Boy. I know this is possible, because Pokémon Yellow does it.

Channel 3 is a wavetable channel, which means I can define a completely arbitrary waveform (read: sound) and channel 3 will play it for me. The correct approach seems obvious: slice the sound into small chunks and ask channel 3 to play them in sequence.

How hard could this possibly be?

Channel 3

Channel 3 plays a waveform from waveform RAM, which is a block of 16 bytes in register space, from $FF30 through $FF3F. Each nybble is one sample, so I have 32 samples whose values can range from 0 to 15.

32 samples is not a whole lot; remember, a common audio rate is 44100 Hz. To keep that up, I’d need to fill the buffer almost 1400 times per second. I can use a lower sample rate, but what? I guess I’ll figure that out later.

First things first: I need to take my sound and cram it into this format, somehow. Here’s the sound I’m starting with.

The original recording was a bit quiet, so I popped it open in Audacity and stretched it to max volume. I only have 4-bit samples, remember, and trying to cram a quiet sound into a low bitrate will lose most of the detail.

(A very weird thing about sound is that samples are really just measurements of volume. Every feature of sound is nothing more than a change in volume.)

Now I need to turn this into a sequence of nybbles. From previous adventures, I know that Python has a handy wave module for reading sample data directly from a WAV file, and so I wrote a crappy resampler:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
import wave

TARGET_RATE = 32768

with wave.open('aowr.wav') as w:
    nchannels, sample_width, framerate, nframes, _, _ = w.getparams()
    outdata = bytearray()
    gbdata = bytearray()

    frames_per_note = framerate // TARGET_RATE
    nybble = None
    while True:
        data = w.readframes(frames_per_note)
        if not data:
            break

        n = 0
        total = 0
        # Left and right channels are interleaved; this will pick up data from only channel 0
        for i in range(0, len(data), nchannels * sample_width):
            frame = int.from_bytes(data[i : i + sample_width], 'little', signed=True)
            n += 1
            total += frame

        # Crush the new sample to a nybble
        crushed_frame = int(total / n) >> (sample_width * 8 - 4)
        # Expand it back to the full sample size, to make a WAV simulating how it should sound
        encoded_crushed_frame = (crushed_frame << (sample_width * 8 - 4)).to_bytes(2, 'little', signed=True)
        outdata.extend(encoded_crushed_frame * (nchannels * frames_per_note))

        # Combine every two nybbles together.  The manual shows that the high nybble plays first.
        # WAV data is signed, but Game Boy nybbles are not, so add the rough midpoint of 7
        if nybble is None:
            nybble = crushed_frame + 7
        else:
            byte = (nybble << 4) | (crushed_frame + 7)
            gbdata.append(byte)
            nybble = None

    with wave.open('aowrcrush.wav', 'wb') as wout:
        wout.setparams(w.getparams())
        wout.writeframes(outdata)

with open('build/aowr.dat', 'wb') as f:
    f.write(gbdata)

This is incredibly bad. It integer-divides the original rate by the target rate, so if I try to resample 44100 to 32768, I’ll end up recreating the same sound again.

I don’t know why I started with 32768, either. The resulting data is too big to even fit in a section! Kicking it down to 8192 is a bit better (5 samples to 1, so the real final rate is 8820), but if I get any smaller, too many samples cancel each other out and I end up with silence! I have no idea what I am doing help.

The aowrcrush.wav file sounds a little atrocious, fair warning.

But it seems to be correct, if I open it alongside the original:

Waveforms of the original sound and its bitcrushed form; the latter is very blocky

Crushing it to four bits caused the graph to stay fixed to only 16 possible values, which is why it’s less smooth. Reducing the sample rate made each sample last longer, which is why it’s made up of short horizontal chunks. (I resampled it back to 44100 for this comparison, so really it’s made of short horizontal chunks because each sample appears five times; Audacity wouldn’t show an actual 8192 Hz file like this.)

It doesn’t sound great, but maybe it’ll be softened when played through a Game Boy. Worst case, I can try cleaning it up later. Let’s get to the good part: playing it!

Playing with channel 3

Here we go! First the global setup stuff I had before.

1
2
3
4
5
6
7
8
9
    ; Enable sound globally
    ld a, $80
    ldh [rAUDENA], a
    ; Map instruments to channels
    ld a, $44
    ldh [rAUDTERM], a
    ; Set volume
    ld a, $77
    ldh [rAUDVOL], a

Then some bits specific to channel 3.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
    ld a, $80
    ldh [rAUD3ENA], a
    ld a, $ff
    ldh [rAUD3LEN], a
    ld a, $20
    ldh [rAUD3LEVEL], a
SAMPLE_RATE EQU 8192
CH3_FREQUENCY set 2048 - 65536/(SAMPLE_RATE / 32)
    ld a, LOW(CH3_FREQUENCY)
    ldh [rAUD3LOW], a
    ld a, $80 | HIGH(CH3_FREQUENCY)
    ldh [rAUD3HIGH], a

Channel 3 has its own bit for toggling it on or off in AUD3ENA (NR30); none of the other bits are used. The other new register is AUD3LEVEL (NR32), which is sort of a global volume control. The only bits used are 6 and 5, which make a two-bit selector. The options are:

  • 00: mute
  • 01: play nybbles as given
  • 10: play nybbles shifted right 1
  • 11: play nybbles shifted right 2

Three of those are obviously useless, so 01 it is! That’s where I get the $20.

Figuring out the frequency is a little more clumsy. I used some rgbasm features here to do it for me, and it took a bit of fiddling to get it right. For example, why am I using 65536 instead of 131072, the factor I said was used for the square wave?

The answer is that for the longest time I kept getting this absolutely horrible output, recorded directly from mGBA:

I had no idea what this was supposed to be. Turns out it’s, well, roughly what happens when you halve the Game Boy’s idea of frequency. I finally found out this coefficient was different from the gbdev wiki. I’m guessing the factor of 2 has something to do with there being two nybbles per byte?

Then there’s the division by 32, which neither the manual nor the gbdev wiki mention. The frequency isn’t actually the time it takes to play one sample, but the time it takes to play the entire buffer. Which does make some sense — the “normal” use for the channel 3 is as a custom instrument, so you’d want to apply the frequency to the entire waveform to get the right notes out. This was even more of a nightmare to figure out, since it produced… well, mostly just garbage. I’ll leave it to your imagination.

1
2
3
4
    ld a, 256 - 4096 / (SAMPLE_RATE / 32)
    ldh [rTMA], a
    ld a, 4
    ldh [rTAC], a

Oho! TMA and TAC are new.

The CPU has a timer register, TIMA, which counts up every… well, every so often. It’s only a single byte, and when it overflows, it generates a timer interrupt. It then resets to the value of TMA.

TAC is the timer controller. Bit 2 enables the timer, and the lower two bits select how fast the clock counts up.

Above, I’m using clock speed 00, which is 4096 Hz. The expression for TMA computes SAMPLE_RATE / 32, which is the number of times per second that the entire waveform should play, and then divides that into 4096 to get the number of timer ticks that the waveform plays for. Subtract that from 256, and I have the value TIMA should start with to ensure that it overflows at the right intervals.

I note that this will cause a timer interrupt 256 times per second, which sounds like a lot on a CPU-constrained system. It’s only 4 or 5 interrupts per frame, though, so maybe it won’t intrude too much. I’ll burn down that bridge when I come to it.

Now I just need to enable timer interrupts:

1
2
3
4
start:
    ; Enable interrupts
    ld a, IEF_TIMER | IEF_VBLANK
    ldh [rIE], a

And of course do a call in the timer interrupt, which you may remember is a fixed place in the header:

1
2
3
SECTION "Timer overflow interrupt", ROM0[$0050]
    call update_aowr
    reti

One last gotcha: I discovered that timer interrupts can fire during OAM DMA, a time when most of the memory map is inaccessible. That’s pretty bad! So I also added di and ei around my DMA call.

Okay! I’m so close! All that’s left is the implementation of update_aowr.

Updating the waveform

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
aowr:
INCBIN "build/aowr.dat"
aowr_end:

; ...

update_aowr:
    push hl
    push bc
    push de
    push af

    ; The current play position is stored in music_offset, a
    ; word in RAM somewhere.  Load its value into de
    ld hl, music_offset
    ld d, [hl]
    inc hl
    ld e, [hl]

    ; Compare this to aowr_end.  If it's >=, we've reached the
    ; end of the sound, so stop here.  (Note that the timer
    ; interrupt will keep firing!  This code is a first pass.)
    ld hl, aowr_end
    ld a, d
    cp a, h
    jr nc, .done
    jr nz, .continue
    ld a, e
    cp a, l
    jr nc, .done
    jr z, .done
.continue:

    ; Copy the play position back into hl, and copy 16 bytes
    ; into waveform RAM.  This unrolled loop is as quick as
    ; possible, to keep the gap between chunks short.
    ld h, d
    ld l, e
_addr = _AUD3WAVERAM
    REPT 16
    ld a, [hl+]
    ldh [_addr], a
_addr = _addr + 1
    ENDR

    ; Write the new play position into music_offset
    ld d, h
    ld e, l
    ld hl, music_offset
    ld [hl], d
    inc hl
    ld [hl], e
.done:
    pop af
    pop de
    pop bc
    pop hl
    ret

Perfect! Let’s give it a try.

Hey, that’s not too bad! I can see wiring that up to a button and pressing it relentlessly. It’s a bit rough, but it’s not bad for this first attempt.

That was mGBA, though, and I’ve had surprising problems before because I was reading or writing when the actual hardware wouldn’t let me. I guess it wouldn’t hurt to try in bgb. (warning: very bad)

OH NO

What has happened.

Tragedy

A lot of fussing around, reading about obscure trivia, and being directed to SamplePlayer taught me a valuable lesson: you cannot write to waveform RAM while the wave channel is playing.

Okay. No problem. I’ll just turn it off, write to wave RAM, then turn it back on. Turning it off clears the frequency, but that’s fine, I can just write it again.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
    ; Disable channel 3 to allow writing to wave RAM
    xor a
    ldh [rAUD3ENA], a

    ; ... do the copy ...

    ld a, $80
    ldh [rAUD3ENA], a
    ld a, LOW(CH3_FREQUENCY)
    ldh [rAUD3LOW], a
    ld a, $80 | HIGH(CH3_FREQUENCY)
    ldh [rAUD3HIGH], a

Okay! Perfect! I’m so ready for a meow!!!

why god why

This is what I get in mGBA and SameBoy. Ironically, it plays fine in bgb.

It seems I have come to an impasse.

Why

After a Herculean amount of debugging and discussion with people who actually know what they’re talking about, here’s what I understand to be happening.

When the wave channel first starts playing, it doesn’t correctly read the very first nybble; instead, it uses the high nybble of whatever was already in its own internal buffer.

Disabling the wave channel sets its internal buffer to all zeroes.

I disable the wave channel every time it plays. Effectively, every 32nd sample starting with the first is treated as zero, which is the most extreme negative value, which is why the playback looks like this (bearing in mind that mGBA’s audio is currently upside-down):

The above sound's waveform, which resembles the original, but with regularly spaced spikes

For whatever reason, bgb doesn’t emulate this spiking, so it plays fine. I’m told the spiking also happens on actual hardware, but the speakers are cheap so it’s harder to notice.

SamplePlayer isn’t much help here, because it’s subject to the same problem.

A ray of hope, dashed

But wait! There’s one last thing I can try. Pokémon Yellow has freeform sounds in it, and it doesn’t have this spiking! There’s even a fan disassembly of it!

Alas. Pokémon Yellow doesn’t use channel 3 to play back sounds. It uses channel 1.

How, you ask? Remember when I said earlier that hearing is really just detecting changes in volume? Pokémon Yellow plays a constant square wave and simply toggles it on and off, very rapidly. Channel 3 is 4-bit; the sounds Pokémon Yellow plays are 1-bit, on or off. It’s baffling, but it does work.

I don’t think it’ll work for me, since that means 32 times as many interrupts. In fact, Pokémon Yellow uses a busy loop as a timer, so it effectively freezes the entire rest of the game anytime it plays a Pikachu sound. I’d rather not do that, but… I don’t seem to have a lot of options.

And so I’ve reached a dead end. The spiking seems to be a fundamental bug with the Game Boy sound hardware. I’ve found evidence that it may even still exist in the GBA, which uses a superset of the same hardware. I can’t fix it, I don’t see how to work around it, and it sounds really incredibly bad.

After days of effort trying to get this to work, I had to shelve it.

The title of this post is a sort of pun, you see, a play on words—

To be continued

This work doesn’t correspond to a commit at all; it exists only as a local stash.

Next time: dialogue! And this time it works!

Cheezball Rising: Maps and sprites

Post Syndicated from Eevee original https://eev.ee/blog/2018/07/15/cheezball-rising-maps-and-sprites/

This is a series about Star Anise Chronicles: Cheezball Rising, an expansive adventure game about my cat for the Game Boy Color. Follow along as I struggle to make something with this bleeding-edge console!

GitHub has intermittent prebuilt ROMs, or you can get them a week early on Patreon if you pledge $4. More details in the README!


In this issue, I get a little asset pipeline working and finally have a real map.

Previously: spring cleaning.
Next: resounding failure.

Recap

The last post only covered some minor problems (including, I grant you, being totally broken), so the current state of the game is basically unchanged from before.

A space cat roams around on a grassy background

That grass pattern, the grass sprite itself, and the color scheme are all hardcoded — written directly into the source code, by hand. If this game is going to get very far at all, I urgently need a better way to inject some art.

Constraints

The Game Boy imposes some fairly harsh constraints on the artwork — which is part of the charm! But now I have to figure out how to work within those constraints most effectively. Here’s what I’ve got to work with.

Bear in mind that I intend for the game to be based around 16×16, um, tiles. Okay, it’s extremely confusing that “tile” might refer either to the base size of the artwork or to the Game Boy’s native 8×8 tiles, so I’m going to call the art tiles and the Game Boy’s basic unit a character (which is what the manual does).

  • The background layer is a grid of 8×8 characters, each of which uses one of eight 4-color background palettes.

  • The object layer is a set of 8×16 character pairs, each of which uses one of eight 3-color object palettes. These palettes are 3-color because color 0 is always transparent.

  • No more than 40 objects can appear on screen at the same time. (There is a way to weasel past this limit, but it requires considerable trickery.)

  • No more than 10 objects can appear in the same row of pixels. (I believe this is a hard limit.)

  • There are three blocks of 256 chars each. I can divide this between the background and objects more or less however I want, though neither can have more than two blocks (= 512 chars).

I’m intending for the game to be based around a 16×16 grid, a fairly common size for the Game Boy. That makes me a little concerned about the per-row object limit — each entity will need to have two Game Boy objects side by side, so I’m really limited to only five entities sharing the same row of pixels. I can’t do much about that quite yet (and only have one entity anyway), but it’s likely to affect how I design maps and draw sprites.

The next biggest problem is colors. Each object palette can only have three colors, which in practice means a shadow/outline color, a highlight color, and a base color. This is why every NPC and overworld critter in Pokémon GSC and the Zeldas is basically monochromatic. They pull it off really well by making very effective use of the highlight and shadow colors.

Since 16×16 sprites are composed of multiple Game Boy objects, it’s possible to overcome this limit by giving each part of the sprite a different palette. Unfortunately, objects being 8×16 means the sprites are split vertically, when it would be most useful to have different colors for e.g. the head and body. I wish the Game Boy supported 16×8 objects! That’d help a ton with the per-row limit, too. Alas, a few decades too late to change it now.


As for the number of chars… well, let’s see. The whole screen is only 160×144, which is 20×18 or 360 chars, so I could allocate two blocks to the background and have 512 — more than enough to cover the entire screen in unique chars! (I expect one block to be more than enough for objects, since I can only show 80 object chars at once anyway.)

On the other hand, I’ll need to reserve some of that space for text and UI and whatnot, and each 16×16 tile is composed of four chars. If I very generously allocate a whole block to window dressing (enough for all of ISO-8859-1?), that leaves 256 chars, which is 64 tiles, which is a tileset that fits in an eight-by-eight square.

For comparison’s sake, even fox flux’s relatively limited tileset is a sixteen-tile square — four times as big. This feels a little dire.

But how can it be dire, when I have enough sprite space to fill the screen and then some?

Let’s see here. A pretty good chunk of the fox flux tileset is unused or outright blank. Some of these tiles are art for moving objects that happened to fit in the grid, and those wouldn’t be in the background tileset. And while all of the tiles are distinct, a lot of the basic terrain has some significant overlap:

A set of dirt tiles from fox flux, colored to indicate where different tiles have identical corners

All of the regions of the same color are identical. These 9 distinct tiles could fit into 20 chars if they shared the common parts, rather than the 36 required to naïvely cutting each one into four dedicated chars.

(The fox flux grid is 32×32, so everything is twice as big as it will be on the Game Boy, but you get the idea.)

I’m feeling a little better about this, especially knowing I do have enough space to cover the whole screen. Worst case, I could draw the map as though it were a single bitmap. I don’t want to have to rely on that if I can get away with it, though — I suspect I’d need to constantly load chars on the fly, and copying stuff around eats into my CPU budget surprisingly quickly.

Research

That does get me wondering: what, exactly, do the Oracle games do? I haven’t done any precise measurements, but I’m pretty sure they have more than sixty-four distinct map tiles throughout their large connected worlds. Let’s have a look!

Oracle of Ages and its live tilemap, in the graveyard, showing the graveyard tileset

Here I am in the graveyard near the start of Oracle of Ages. The “creepy tree” here is distinct and doesn’t really appear anywhere else, so I found it in the tile viewer (lower right) and will be keeping an eye on it. Note that only the left half of the face is visible; the right half is using the same tiles, flipped horizontally. (The colors are different because the tile viewer shows the literal colors, whereas the game itself is being drawn with a shader.)

Let’s walk left one screen.

Oracle of Ages and its live tilemap, outside of the graveyard

Now, this is interesting. The creepy tree is still on the screen here, so its tiles are naturally still loaded. But a bunch of tiles on the left — parts of the dungeon entrance and other graveyard things — have been replaced by town tiles. I’m several screens away from the town!

The next screen up has no creepy trees, but its tiles remain. Of course, they’d have to, since the creepy tree is still visible during a transition. I have to go left from there before the tree disappears:

Oracle of Ages and its live tilemap, with tiles spelling SHOP clearly visible

Wow! At a glance, this looks like enough tiles to draw the entire town.

This is fascinating. The Oracle games have several transitions between major areas, marked by fade-outs or palette changes — the purple-tinted graveyard is an obvious example. But it looks like there are also minor transitions that update the tileset while I’m still several screens away from where those tiles are used. The screens around the transition only use common tiles like grass and regular trees, so I never notice anything is happening.

That’s cute, clever, and an easy way to make screen transitions work without having to figure out what tiles are becoming unused as they slide off the screen!

At this point I realize I may be getting ahead of myself. Screen transitions? I don’t have a map yet! Hell, I don’t even have a camera. Time to back up and make something I can build on.

Designing a tileset

I’m pretty tired of manually translating art into bits. It’s 2018, dammit. I want to use all the regular tools I would use for this, I want the Game Boy’s limitations to be expressed as simply as possible, and I want minimal friction between the source artwork and the game.

Here’s my idea. I know I only have 8 palettes to work with, so I’m decreeing that tilesets will be stored as paletted PNGs. The first four colors in the image palette will become the first Game Boy palette; the next four colors become the second Game Boy palette; and so on. If I then resize Aseprite’s palette panel to be four colors wide, I’ll have an instant view of all my available combinations of colors.

This already has some problems — for starters, if the same color appears in multiple palettes (which will almost certainly happen, for the sake of cohesion), I’m very likely to confuse the hell out of myself. I also have no idea how to extend this into multiple tilesets, but for now I’ll pretend the entire game world only uses a single tileset.

I could instead dynamically infer the palettes based on what combinations of colors are actually used, but after more than a couple tiles, it would be a nightmare for a human to keep track of what those combinations are. With this approach, all a human needs to do is color-drop a pixel from a particular tile and look at what row the color’s in.

After a quick jaunt into the pixel mines, here are some tiles.

A small set of pastel yellow moon tiles

Or, as viewed in Aseprite:

The same set of tiles, as seen in an editor, with the four-color palette visible

That’s only one palette, but hopefully you can see what I’m going for here. It’s enough to get started.

At this point, I started writing a little Python script that used Pillow to inspect the colors and pixels and dump them out to rgbasm-flavored source code. The script itself is not especially interesting: run through each 8×8 block of pixels, look at each pixel’s palette index, mod 4 to get the index within the Game Boy palette, print out as backtick literals. (I could spit out raw binary data, but I wanted to be able to inspect the intermediate form easily. Maybe later.)

The results:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
SECTION "Map dumping test", ROM0
TEST_PALETTES:
    dw %0101011110111101
    dw %0101011100011110
    dw %0100101010111100
    dw %0100011001111000
    ; ... enough zeroes to make eight palettes ...
; sorry, in the script I was calling them "tiles", not "chars"
TEST_TILES:
    ; tile 0 at 0, 0
    dw `00001000
    dw `00000000
    dw `00100000
    dw `00000000
    dw `00000000
    dw `00000000
    dw `20000000
    dw `20000002
    ; ... etc ...

And hey, I already have code that can load palettes and chars, so all I have to do is swap out the old labels for these ones.

Now I have a tileset I can load into the game, which is very exciting, except that I can’t see any of them because I still don’t have a map. I could draw a test map by hand, I suppose, but the whole point of this exercise was to avoid ever doing that again.

Drawing a map

In keeping with the “it’s 2018 dammit” approach, I elect to use Tiled for drawing the maps. I’ve used it for several LÖVE games, and while its general-purposeness makes it a little clumsy at times, it’s flexible enough to express basically anything.

I make a tileset and create a map. I choose 256×256 pixels (16×16 tiles), the same size as the Game Boy screen buffer, and fill it with arbitrary terrain. In retrospect, I probably should’ve made it the size of the screen, since I still don’t have a camera. Oh, well.

Here, I hit a minor roadblock. I want to do as much work as possible upfront, so I want to store the map in the ROM as chars, not tiles. That means I need to know what chars make up each tile, which is determined by the script that converts the image to char data. Multiple maps might use the same tileset, and a map might use multiple tilesets, so it seems like I’ll need some intermediate build assets with this information…

(In retrospect again, I realize that the game may need to know about tiles rather than just chars, since there’ll surely be at least a few map tiles that act like entities — switches and the like — and those need to function as single units. I guess I’ll work that out later.)

This is all looking like an awful lot of messing around (and a lot of potential points of failure) before I can get anything on the dang screen. I waffle for a bit, then decide to start with a single step that simultaneously dumps the tiles and the map. I can split it up when I actually have more than one of either.

You can check out the resulting script if you like, but again, I don’t think it’s particularly interesting. It enforces a few more constraints than before, and adds a TEST_MAP_1 label containing all the char data, row by row. Loading that into VRAM is almost comically simple:

1
2
3
4
5
    ; Read from the test map
    ld hl, $9800
    ld de, TEST_MAP_1
    ld bc, 1024
    call copy16

The screen buffer is 32×32 chars, or 1024 bytes. As you may suspect, copy16 is like copy, but it takes a 16-bit count in bc.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
; copy bc bytes from de to hl
; NOTE: bc must not be zero
copy16:
    ld a, [de]
    inc de
    ld [hl+], a
    dec bc
    ; dec bc doesn't set flags, so gotta check by hand
    ld a, b
    or a, c
    jr nz, copy16
    ret

Hm. It’s a little harder to justify the bc = 0 case as a feature here, since that would try to overwrite every single byte in the entire address space. Don’t do that, then.

Anise, in-game, walking on the moon tiles

Now, at long long last, I have a background with some actual art! It’s starting to feel like something! I’ve even got something resembling a workflow.

My desktop, showing the moon tiles in an image editor, the map put together in Tiled, and the game running in mGBA

All in a day’s work. Good time to call it, right?

Except

I just wrote this char loading code…

And there’s still one thing still hardcoded…

I wonder if I could do something about that…?

Sprites

Above, I conspicuously did not mention how I integrated the Python script into the build system. And, well, I didn’t do that. I ran it manually and put it somewhere and committed it all as-is. You currently (still!) can’t actually build the game without repeating my steps. You can’t even just put the output in the right place, because you also have to delete some debug output from the middle of the file.

It gets worse! Here’s how.

I have some Anise walking sprites, too, drawn in Aseprite. They’re pretty cute and I’d love to have them in the game, now that I have some Real Art™ for the background.

Star Anise, walking forwards

Why not throw these at the same script and hack them into animating?

Unfortunately, this introduces a bit of manual work, as animation often does. (My kingdom for a way to embed a small simple animation in a larger spritesheet in Aseprite!) I’ve typically animated every critter in its own Aseprite file — or stacked several vertically in the same file when their animations are similar enough — and then exported as a sheet with the frames running off horizontally. You can see this at work in fox flux, e.g. on its critter sheet.

But Star Anise introduces a wrinkle that prevents even that slightly clumsy workflow from working.

You may have noticed that the walking sprite above blows the color budget considerably, using a whopping five colors. The secret is that Anise himself fits in a 16×16 square, and then his antenna is a third 8×16 sprite drawn on top. I can’t simply export him as a spritesheet, because the antenna needs to be separate, and it’s not even aligned to the grid. It doesn’t even stay in the same place consistently!

I could maybe hack something together that would automatically pull the incompatible pixels into a separate sprite. I might need to, since — spoiler alert — there are an awful lot of Lunekos in this game. For now, though, I did the dumbest thing that works and copied his frames to their own sheet by hand.

Star Anise's walking frames laid out in a spritesheet

The background is actually cyan, not transparent. I had to do this because my setup expects multiple sets of four colors — the first color in an object palette is still there, even if it’s ignored — and only one color in an indexed PNG can be transparent. (Don’t @ me about PNG pixel formats.) I could’ve adjusted it to work with sets of three colors and put the transparent one at the end so the palette column trick still worked, but… this was easier.

Here’s the best part: I took the main function from my tile loading script, copy-pasted it within the same file, and edited the copy to dump these sprites sans map. So now not only is there no build system, but half of the loading script is inaccessible! Sorry. We’re getting into experiment territory and I am going to start making a lot of messes while I figure out what I actually want.

Using these within the game was just as easy as before — replace some labels with new ones — and the only real change was to use a third OAM slot for the antenna. (The antenna has to appear first; when sprites overlap, the one with the lowest index appears on top.)

That did make updating OAM a little clumsy; you may recall that before, I loaded the x and y positions into b and c, updated them, then wrote them back into OAM:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    ; set b/c to the y/x coordinates
    ld hl, oam_buffer
    ld b, [hl]
    inc hl
    ld c, [hl]
    bit BUTTON_LEFT, a
    jr z, .skip_left
    dec c
.skip_left:
    bit BUTTON_RIGHT, a
    jr z, .skip_right
    inc c
.skip_right:
    bit BUTTON_UP, a
    jr z, .skip_up
    dec b
.skip_up:
    bit BUTTON_DOWN, a
    jr z, .skip_down
    inc b
.skip_down:
    ld [hl], c
    dec hl
    ld [hl], b
    ld a, c
    add a, 8
    ld hl, oam_buffer + 5
    ld [hl], a
    dec hl
    ld [hl], b

The above approach required that I hardcode the 8-pixel offset between the left and right halves. With the antenna in the mix, I would’ve had to hardcode another more convoluted offset, and I didn’t like the sound of that. So I changed it to inc and dec the OAM coordinates directly and immediately:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
    ; Anise update loop
    ; set b/c to the y/x coordinates
    ld bc, 4
    bit BUTTON_LEFT, a
    jr z, .skip_left
    ld hl, oam_buffer + 1
    dec [hl]
    add hl, bc
    dec [hl]
    add hl, bc
    dec [hl]
.skip_left:
    ; ... etc ...

Eventually I should stop doing this and have an actual canonical x/y position for Anise somewhere. But I didn’t do that yet.

I did also take this opportunity to change my LCDC flags so that object chars start counting from zero at $9000, fixing the misunderstanding I had before. That’s nice.

Anyway, tada, Star Anise can slide around, but now with his antenna.

Not good enough.

Animating

It’s time to animate something. And this time around, all I’ve got are bytes to work with. Oh, boy!

Right out of the gate, I have two options. I could load all of Anise’s sprites into VRAM upfront and change the char numbers in OAM to animate him, or I could reserve some specific chars and overwrite them to animate him.

The first choice makes sense for an entity that might exist multiple times at once, like enemies or… virtually anything in the game world, really. But there’s only ever one player, and he’s likely to have a whole lot of spritework, which I would prefer not to have clogging up my char space for the entire duration of the game. So while I might use the other approach for most other things, I’m going to animate Anise by overwriting the actual graphics. Every frame.

First things first. I’m going to need some state, which I’ve been avoiding by relying on OAM. At the very least, I need to know which way Anise is facing — which isn’t necessarily the direction he’s moving, because he should keep his facing when he stops. I also need to know which animation frame he’s on, and how many LCD frames are left until he should advance to the next one.

Let’s refer to the time between vblanks as a “tic” for now, to avoid the ambiguity of a “frame” when talking about animation.

A good start, then, would be some constants.

1
2
3
4
5
6
FACING_DOWN   EQU 0
FACING_UP     EQU 1
FACING_RIGHT  EQU 2
FACING_LEFT   EQU 3

ANIMATION_LENGTH EQU 5

ANIMATION_LENGTH is the length of every frame. I don’t especially want to give every frame its own distinct duration if I can avoid it; this will be complicated enough as it is. I fiddled with the frame duration in Aseprite for a bit and landed on 83ms as a nice speed, and that’s 5 tics.

I also need a place for this state, so I add some more stuff to my RAM block.

1
2
3
4
5
6
anise_facing:
    db
anise_frame:
    db
anise_frame_countdown:
    db

And initialize it in setup.

1
2
3
4
    ld a, FACING_DOWN
    ld [anise_facing], a
    ld a, ANIMATION_LENGTH
    ld [anise_frame_countdown], a

Presumably, one day, I’ll have multiple entities, and they’ll all share a similar structure, which I’ll have to traverse manually. For now, it’s easier to follow the code if I give every field its own label.

I have four levels of hierarchy here: the spriteset (which for now is always Anise’s), the pose (I only have one: walking), the facing, and the frame. I need to traverse all four, but luckily I can ignore the first two for now.

I don’t want to animate Anise when he’s not moving, so I changed the OAM updating code to also ld d, 1 if there’s any movement at all, and skip over all the animation stuff if d is still zero.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
    ; ... read input ...

    ; This was before I knew the 'or a' trick; these two ops
    ; could be replaced with 'xor a; or d'
    ld a, d
    cp a, 0
    jp z, .no_movement

    ; ... all the animation code will go here ...

.no_movement:
    ; and after this we repeat the main loop

This does have the side effect that Anise will simply freeze in mid-walk when stopped, rather than returning to his standing pose. I still haven’t fixed that; I could special-case it, but I usually treat “standing” as its own one-frame animation, so it feels like something that ought to come when I implement poses.

Next I decrement the countdown, which is the number of tics left until the frame ought to change. If this is nonzero, I don’t need to do anything.

1
2
3
4
5
6
    ld a, [anise_frame_countdown]
    dec a
    ld [anise_frame_countdown], a
    jp nz, .no_movement
    ld a, ANIMATION_LENGTH
    ld [anise_frame_countdown], a

Again, this isn’t actually right. If Anise’s state changes, such as between standing and walking, then this should be ignored because he’s switching to a new animation. But this is a pose thing again, so I’m deferring it until later.

Next I need to advance the current frame. I don’t have modulo on hand and even simple ifs are kind of annoying, so I was naughty here and used bitops to roll from frame 3 to frame 0. This would obviously not work if the number of frames were not a power of two.

1
2
3
4
    ld a, [anise_frame]
    inc a
    and a, 4 - 1
    ld [anise_frame], a

Yet again, if Anise changes direction, the frame should be reset to zero… but it ain’t.

Now, let’s think for a second. I know what frame I want. I have a label for the upper-left corner of the spritesheet, and I want to get to the upper-left corner of the appropriate frame. Each frame has 3 objects; each object has 2 chars; each char is 16 bytes.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
    ld hl, ANISE_TEST_TILES
    ; Skip ahead 3 sprites * the current frame
    ld bc, 3 * 2 * 16
    ; Remember, zero iterations is also possible
    or a
    jr z, .skip_advancing_frame
.advance_frame:    
    add hl, bc
    dec a
    jr nz, .advance_frame
.skip_advancing_frame:
    ; Copy the sprites into VRAM
    ; They're consecutive in both the data and VRAM, so only
    ; one copy is necessary.  And bc is already right!
    ld d, h
    ld e, l
    ld hl, $8000
    call copy16

Hey, look at that!

Star Anise walking around in-game, now animated

Only one small problem: I forgot about facing, so Anise will always face forwards no matter how he moves. Whoops!

Facing

I need to actually track which way Anise is facing, which is a surprisingly subtle question. He might even be facing away from his own direction of movement, if for example he was thrown backwards by some external force.

A decent first approximation is to use the last button that was pressed. (That’s still not quite right — if you hold down, hold down+right, and then release right, he should obviously face down. But it’s a start.)

I don’t yet track which buttons were pressed this frame, but it’s easy enough to add. While I’m at it, I might as well track which buttons were released, too. I amend the input reading code thusly, based on the straightforward insight that a button was pressed this frame iff it is currently 1 and was previously 0.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
    ; a now contains the current buttons
    ld hl, buttons
    ld b, [hl]                  ; b <- previous buttons
    ld [hl], a                  ; a -> current buttons
    cpl
    and a, b
    ld [buttons_released], a    ; a = ~new & old, i.e. released
    ld a, [hl]                  ; a <- current buttons
    cpl
    or a, b
    cpl
    ld [buttons_pressed], a     ; a = ~(~new | old), i.e. pressed

I like that cute trick for getting the pressed buttons. I need a & ~b, but cpl only works on a, so I would’ve had to juggle a bunch of registers. But applying De Morgan’s law produces ~(~a | b), which only requires complementing a. (Full disclosure: I didn’t actually try register juggling, and for all I know it could end up shorter somehow.)

Next I check the just-pressed buttons and updating facing accordingly. It looks a lot like the code for checking the currently-held buttons, except that I only use the first button I find.

1
2
3
4
5
6
7
8
    ld hl, anise_facing
    ld a, [buttons_pressed]
    bit BUTTON_LEFT, a
    jr z, .skip_left2
    ld [hl], FACING_LEFT
    jr .skip_down2
.skip_left2:
    ; ... you get the idea ...

And finally, amend the sprite choosing code to pick the right facing, too.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
    ld hl, ANISE_TEST_TILES

    ; Skip ahead a number of /rows/, corresponding to facing
    ld a, [anise_facing]
    and a, %11                      ; cap to 4, just in case
    jr z, .skip_stride_row
    ; This is like before, but times 4 frames
    ld bc, 4 * 3 * 2 * 16
.stride_row:
    add hl, bc
    dec a
    jr nz, .stride_row
.skip_stride_row:

    ; Bumping the frame here is convenient, since it leaves the
    ; frame in a for the next part
    ld a, [anise_frame]
    inc a
    and a, 4 - 1
    ld [anise_frame], a

    ; ... continue on with picking the frame ...

Hardcoding the number of frames here is… unfortunate. I should probably flip the spritesheet so the frames go down and each column is a facing; then there’ll always be a fixed number of columns to skip over.

But who cares about that? Look at Anise go! Yeah!

Star Anise walking around in-game, now animated in all four directions

Well, yes, there is one final problem, which is that the antenna is misaligned when walking left or right… because its positioning is different than when walking up or down, and I don’t have any easy way to encode that at the moment. It’s still like that, in fact. I’m sure I’ll fix it eventually.

More vblank woes

I didn’t run into this problem until a little while later, but I might as well mention it now. The above code writes into VRAM in the middle of updating entities — updating them very simply, perhaps, but updating nonetheless. If that updating takes longer than vblank, the write will fail.

I expected this, though not quite so soon. It’s a disadvantage of swapping the char data rather than the char references: 32× more writing to do, which will take 32× longer. The solution is similar to what I do for OAM: defer the write until the next vblank. I’m already doing that with Anise’s position, anyway, and it makes no sense to have his position and animation updated on different frames.

I ended up special-casing this for Anise, though it wouldn’t be too hard to extend this into a queue of tiles to copy. It’s nothing too world-shaking; I just store the address of Anise’s current sprite in RAM, then copy it over during vblank, just after the OAM DMA.

I did try doing this with one of the Game Boy Color’s new features, general-purpose DMA, which can copy from basically anywhere in ROM or RAM to basically anywhere in VRAM. It involves five registers: you write the source address in the first two, the destination in the next two, and the length in the fifth, which triggers the copy. The CPU simply freezes until the copy is done, so there are no goofy timing issues here.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
    ld hl, anise_sprites_address
    ld a, [hl+]
    ld [rHDMA1], a
    ld a, [hl]
    ld [rHDMA2], a
    ld a, HIGH($0000)
    ld [rHDMA3], a
    ld a, LOW($0000)
    ld [rHDMA4], a
    ; To copy X bytes, write X / 16 - 1 to this register
    ld a, (32 * 3) / 16 - 1
    ld [rHDMA5], a

General-purpose DMA can copy 16 bytes every 8 cycles, or ½ cycle per byte. The fastest possible manual copy would be an unrolled series of ld a, [hl+]; ld [bc], a; inc bc which takes a whopping 6 cycles per byte — twelve times slower! This is a neat feature.

FYI, it’s also possible to have a copy done piecemeal during hblanks, though that sounds a bit fragile to me.

Future work

I’ve laid some very basic groundwork here, and there’s plenty more to do, which I will get back to later! It’s just me hacking all this together, after all, and I like flitting between different systems.

I will definitely need to figure out how the heck multiple tilesets work and when they get switched out. How do I even use multiple tilesets, each with its own set of palettes? What’s the workflow if I want to use the same tiles with several different palettes, like how the graveyard in Oracle of Ages is tinted purple? And I didn’t even implement character de-duplication yet… which will require some metadata for each tile… aw, geez.

And I still haven’t fixed the build system! Maybe you can understand why I’m hesitant to impose more structure on this idea quite yet.

To be continued

That brings us to commit 59ff18. Except for a commit about the build that I skipped. Whatever. This post has been a little more draining to write, perhaps because it forced me to confront and explain a bunch of hokey decisions.

Next time: resounding failure!

Cheezball Rising: Spring cleaning

Post Syndicated from Eevee original https://eev.ee/blog/2018/07/13/cheezball-rising-spring-cleaning/

This is a series about Star Anise Chronicles: Cheezball Rising, an expansive adventure game about my cat for the Game Boy Color. Follow along as I struggle to make something with this bleeding-edge console!

GitHub has intermittent prebuilt ROMs, or you can get them a week early on Patreon if you pledge $4. More details in the README!


In this issue, I tidy up some of the gigantic mess I’ve made thusfar.

Previously: writing a main loop, and finally getting something game-like.
Next: sprite and map loading.

Recap

After only a few long, winding posts’ worth of effort, I finally have a game, if you define “game” loosely as a thing that reacts when you press buttons.

A space cat roams around on a grassy background

Beautiful. But to make an omelette, you need to break a few eggs, and if it’s your first omelette then you might break some glassware too. As tiny as this game is, a couple things could use improvement.

Also, for narrative purposes, it’s much more interesting to put all these miscellaneous fixes together, rather than interrupting other posts with them. I didn’t actually do all this work in one lump in this order. Apologies to the die-hard non-fiction crowd.

It’s totally broken

Ah, the elephant in the room. The end of the previous post aligned with the first demo build, but if you downloaded it and tried to play it, you may have seen something that looks more like this:

Similar to the previous image, but with obvious graphical corruption

I said in the beginning that I liked mGBA and would be developing against it. That’s still true — it’s open source (and I’ve actually read some of it), it’s cross-platform, and it has some debug tools built in.

I also said that emulators are primarily designed to accept correct games, not necessarily to reject incorrect games. And that’s still very true.

I discovered this problem myself a little later (after the events of the next post), while shopping around a bit for emulators explicitly focused on accuracy. The one I keep being told to use is bgb, but it’s for Windows and Wine is kind of annoying, so I was exploring my other options; I found SameBoy (primarily for Mac, but with Linux and Windows builds sans debug features) and Gambatte (cross-platform, and the core for RetroArch’s Game Boy emulation). All three of them looked like the screenshot above.

Something was going very wrong when writing to VRAM. You can’t write to VRAM while the LCD is redrawing, so the most obvious cause is that… well… maybe the LCD is redrawing during my setup code.

Remember, on an actual Game Boy, the system doesn’t immediately start running what’s on the cartridge — it scrolls in the Nintendo logo first (or on a Color, does a fancier logo with a cool fanfare). That’s done by a tiny internal program called the boot ROM, and the state of the LCD when the boot ROM hands over control is undefined. I’m sure it’s consistent, but it’s not anything in particular, and for all I know it might be when the LCD is halfway through a redraw.

(Side note: I am violating Nintendo’s game submission requirements by consistently referring to it as a “cartridge” when in fact it is properly called a Game Pak. My bad.)

So what we’re seeing above is the result of VRAM becoming locked and unlocked as the LCD draws (remember, after every row is an hblank, during which time VRAM is accessible), while I’m trying to copy blocks of data there. In fact, every emulator I’ve tried shows a slightly different form of corruption, since this problem is very sensitive to timing accuracy. Super interesting!

I could wait for vblank and try to squeeze in all my setup code there, maybe even split across several vblanks. But since this is setup code and doesn’t run during gameplay, there’s a much easier solution: turn the screen off. That’s done with a bit in the LCDC register, which I currently configure at the end of my setup code; all I need to do is move that to the beginning and clear the appropriate bit instead.

1
2
    ld a, %00010111  ; $91 plus bit 2, minus bit 7
    ld [$ff40], a

Then, of course, set it again once I’m done. I did this with a couple macros, since it’s only a few instructions and it seems like the kind of thing I might need again later.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
DisableLCD: MACRO
    ld a, [$ff40]
    and a, %0111111
    ld [$ff40], a
ENDM

EnableLCD: MACRO
    ld a, [$ff40]
    or a, %10000000
    ld [$ff40], a
ENDM

; and, of course, stick an EnableLCD at the end of setup code

Note that when the screen is off, it’s off, and there are no vblank interrupts or anything else that might be triggered by the screen’s behavior. So, you know, don’t wait for vblank while the screen’s off. When the screen turns back on, it immediately starts redrawing from the first row, so don’t try to use VRAM right away either. Finally, on the original Game Boy, do not turn off the screen when it’s not in vblank, or you might physically damage the screen. It’s fine on the Game Boy Color, but… hell, I’m gonna edit this to wait for vblank anyway. Feels kinda inappropriate to abruptly turn off the screen halfway through drawing.

Anyway, that solves my goofy corruption problems, and now the game looks the same on all of these emulators! I also reported this misbehavior, and it’s since been fixed, so recent dev builds of mGBA also correctly render garbage for the first release. See, by not targeting the most accurate emulators, I’ve caused another emulator to become more accurate!

hardware.inc

I mentioned last time that I’d adopted hardware.inc. That’s in large part because I keep producing monstrosities like the previous snippet. Here are those macros with some symbolic constants:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
DisableLCD: MACRO
    ld a, [rLCDC]
    and a, $ff & ~LCDCF_ON
    ld [rLCDC], a
ENDM

EnableLCD: MACRO
    ld a, [rLCDC]
    or a, LCDCF_ON
    ld [rLCDC], a
ENDM

A breath of fresh air!

The $ff & is necessary because the argument needs to fit in a byte, but rgbasm’s integral preprocessor type is wider than a byte. I suppose I could also use LOW() here, or maybe there’s some other more straightforward solution.

Rearranging the buttons

In the previous post, I read the button states and crammed them into a single byte. I had a choice of whether to put the dpad low or the buttons low, but it didn’t seem to matter, so I picked arbitrarily: buttons high, dpad low.

It turns out I chose wrong! Also, it turns out there’s a “wrong” here! I’ve heard two compelling reasons to do it the other way. For one, hardware.inc contains constants for the bit offsets of the buttons, and it assumes the dpad is high. Why is this arbitrary data layout decision embedded in a list of hardware constants? Possibly for the second reason: on the GBA, input is available as a single word, and the lowest byte contains bits for all the buttons on the Game Boy — in the same order, with the dpad high.

So I’m switching this around and using hardware.incs constants. Easy change.

Fixing vblank

My original approach to waiting for vblank seemed simple enough: loop until vblank_flag is set, clear it, then continue on.

I’ve made a slight oversight here: what if the main loop does take longer than a frame? Then a vblank interrupt will fire in the middle of it and harmlessly set vblank_flag. But when the loop finally finishes and goes to wait for vblank again, the flag will already be set, and it’ll continue on immediately — regardless of the state of the screen! Whoops.

Again, the fix is simple: clear the flag before beginning to wait.

And while I’m at it, I see other uses for waiting for vblank in the near future, so I may as well pull this out into a function.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
; idle until next vblank
wait_for_vblank:
    xor a                       ; clear the vblank flag
    ld [vblank_flag], a
.vblank_loop:
    halt                        ; wait for interrupt
    ld a, [vblank_flag]         ; was it a vblank interrupt?
    and a
    jr z, .vblank_loop          ; if not, keep waiting
    ret

Copy function

So far, I’ve done an awful lot of runtime copying by using the preprocessor. Consider the code for copying the DMA routine into HRAM:

1
2
3
4
5
6
7
8
    ; Copy the little DMA routine into high RAM
    ld bc, dma_copy
    ld hl, $ff80
    REPT dma_copy_end - dma_copy
    ld a, [bc]
    inc bc
    ld [hl+], a
    ENDR

This will repeat the ld/inc/ld dance 13 times in the built ROM. Which is fine, except that I’m about to have places where I do much more copying, and there’s only so much space in the ROM, and this is kind of ridiculous. So I guess I will finally write a copy function.

I’m calling it copy, not memcpy. What else am I going to copy, if not memory?

Attempt number 1 looked like this:

1
2
3
4
5
6
7
8
; copy d bytes from bc to hl
copy:
    ld a, [bc]
    inc bc
    ld [hl+], a
    dec d
    jr z, copy
    ret

I was then informed that it’s more idiomatic to use de as the source address and c as the count, possibly for some reason relating to the NES or SNES? I don’t remember. I’m totally on board for using c to mean a count, though, and started doing that elsewhere.

I went to change that, and actually make use of this function, and lo! I discovered a colossal bug. That last line, jr z, copy, will loop only if d was just decremented to zero. So this function will only ever copy one byte, unless you asked to copy only one byte, in which case it copies two.

This is not the first time I’ve gotten a condition backwards. I’ll get used to it eventually, I’m sure.

Oh, one other minor problem: if you ask to copy zero bytes, you’ll actually copy 256, since the zero check only comes after the decrement. (This is a recurring annoyance, actually, and makes while loops surprisingly clumsy to express.) So far I’ve only ever needed to copy a constant amount, so this hasn’t been a problem, but… I’ll just leave a comment pretending it’s a feature.

1
2
3
4
5
6
7
8
9
; copy c bytes from de to hl
; NOTE: c = 0 means to copy 256 bytes!
copy:
    ld a, [de]
    inc de
    ld [hl+], a
    dec c
    jr nz, copy
    ret

And here it is in action:

1
2
3
4
5
    ; Copy the little DMA routine into high RAM
    ld de, dma_copy
    ld hl, $FF80
    ld c, dma_copy_end - dma_copy
    call copy

Cool.

Of course, this is now significantly slower than the original unrolled version. The original took 13 × (2 + 2 + 2) = 78 cycles; the function adds 6 cycles for the call, 4 cycles for the ret, and 13 × (1 + 3) = 52 for the counting and jumping. As c goes to infinity, the function takes about ⅔ longer than unrolling.

If I feel like it, I could mitigate this somewhat by partially unrolling. First I’d mask off some lower bits of c — say, the lowest two — and copy that many bytes. Now the amount of copying left is a multiple of four, so I could shift c right twice and have another loop that copies four bytes at a time, amortizing the cost of the decrement and jump.

It’s not urgent enough for me to want to bother yet, and it’ll make relatively little difference for small copies like this DMA one, but I’m strongly considering it for copying a 16-bit amount.

Reset vectors

Now I have a couple utility functions like copy and wait_for_vblank. I don’t really care where they go, so I put them in their own SECTION and let the linker figure it out.

It took a while for me to notice where, exactly, the linker had put them: at $0000! These functions are small, and I have nothing explicitly placed before the interrupt handlers (which begin at $0040), so rgblink saw some empty space and filled it.

The thing is, the Game Boy has eight instructions of the form rst $xx that act as fast calls — each one jumps to a fixed low address (a “reset vector”), using less time and space than a call would. And those fixed $xx addresses are… $00, and every eight bytes afterwards.

I don’t have any immediate use for these — eight bytes isn’t a lot, though I guess copy could fit in there — but I probably don’t want arbitrary code ending up where they go, so for now I’ll stub them out like I stubbed out the interrupt handlers.

(I have been advised of one very good use for reset vectors: putting a crash handler at $38. Why? Because rst $38 is encoded as $ff, which is a fairly common byte to encounter if you accidentally jump into garbage. A lot of the Game Boy’s RAM is even initialized to $ff at startup.)

Idioms

I’m still discovering what’s considered idiomatic, but here are a couple tidbits.

The set of instructions is a little scattershot as far as arguments go. Several times early on, I wrote stuff like this:

1
2
3
  ld hl, some_address
  ld a, 133
  ld [hl], a

But I overlooked that there are instructions for both ld [hl], n8 and ld [n16], a, so the above can be reduced to two lines. There’s no such thing as ld [n16], n8, though.

A surprising number of instructions can use [hl] directly as an operand — even inc and dec, combining fetch/mutate/store into a single instruction.

xor a is twice as short and twice as fast as ld a, 0. I mean, we’re talking about a single byte and single cycle here, but no reason not to.

(xor a really means xor a, a, but since every boolean op instruction takes a as the first argument anyway, it can be omitted. I don’t like to omit it in most cases, since xor b doesn’t mention a at all and that seems misleading, but it feels appropriate when combining a with itself.)

or a (equivalently, and a) is a quick way to test whether a is zero, since boolean ops set the zero flag.

Color

This is neither here nor there, but since this post began with emulator differences, here’s another one.

The screen you’re reading this on is almost certainly backlit, but the original Game Boy Color screen was not. A fully white pixel on a Game Boy Color is turned off — it’s the color of the screen itself, in which you can probably see your own reflection.

Which raises a tricky question: what color is that? The game thinks it’s pure white, but the screen was a sort of pale yellow. So how should it be rendered in an emulator, on a modern backlit LCD monitor?

Compounding this problem is that Game Boy Color games can also run on the Game Boy Advance, which showed the colors yet slightly differently. And, of course, even monitors may be calibrated differently, in which case it all goes out the window.

It’s interesting to see different emulators’ opinions of how to render color:

The same screenshot, seen in several different emulators with different color schemes

This is exactly the same ROM. The top left is mGBA out of the box, which shows colors completely unaltered — usually fairly saturated. The top right is mGBA with its “gba-colors” shader enabled, which is supposed to replicate how colors appear on a GBA screen, but seems passingly similar to a GBC too. Then on the bottom are two emulators renowned for their accuracy, here wildly disagreeing with each other.

My Game Boy Color is currently in a box somewhere, and until I can find it, I can’t be sure who’s closer. All of these are perfectly fine interpretations of the same art, though.

I may or may not use the “gba-colors” shader, and may or may not fiddle with mGBA’s color settings over time. If the colors vary a bit in future screenshots, that’s probably why.

To be continued

This post doesn’t really correspond to a particular commit very well, since it’s all little stuff I did here and there. I hope you’ve enjoyed the breather, because it’s all downhill from here. In a good way, I mean. Like a rollercoaster.

Next time: map and sprite loading, which will explain how I got from grass to the moon texture in the screenshots above!

Cheezball Rising: Main loop, input, and a game

Post Syndicated from Eevee original https://eev.ee/blog/2018/07/05/cheezball-rising-main-loop-input-and-a-game/

This is a series about Star Anise Chronicles: Cheezball Rising, an expansive adventure game about my cat for the Game Boy Color. Follow along as I struggle to make something with this bleeding-edge console!

GitHub has intermittent prebuilt ROMs, or you can get them a week early on Patreon if you pledge $4. More details in the README!


In this issue, I fill in the remaining bits necessary to have something that looks like a game.

Previously: drawing a sprite.

Recap

So far, I have this.

A very gaudy striped background with half a cat on top

It took unfathomable amounts of effort, but it’s something! Now to improve this from a static image to something a bit more game-like.

Quick note: I’ve been advised to use the de facto standard hardware.inc file, which gives symbolic names to all the registers and some of the flags they use. I hadn’t introduced it yet while doing the work described in this post, but for the sake of readability, I’m going to pretend I did and use that file’s constants in the code snippets here.

Interrupts

To get much further, I need to deal with interrupts. And to explain interrupts, I need to briefly explain calls.

Assembly doesn’t really have functions, only addresses and jumps. That said, the Game Boy does have call and ret instructions. A call will push the PC register (program counter, the address of the current instruction) onto the stack and perform a jump; a ret will pop into the PC register, effectively jumping back to the source of the call.

There are no arguments, return values, or scoping; input and output must be mediated by each function, usually via registers. Of course, since registers are global, a “function” might trample over their values in the course of whatever work it does. A function can manually push and pop 16-bit register pairs to preserve their values, or leave it up to the caller for speed/space reasons. All the conventions are free for me to invent or ignore. A “function” can even jump directly to another function and piggyback on the second function’s ret, kind of like Perl’s goto &sub… which I realize is probably less common knowledge than how call/return work in assembly.

Interrupts, then, are calls that can happen at any time. When one of a handful of conditions occurs, the CPU can immediately (or, rather, just before the next instruction) call an interrupt handler, regardless of what it was already doing. When the handler returns, execution resumes in the interrupted code.

Of course, since they might be called anywhere, interrupt handlers need to be very careful about preserving the CPU state. Pushing af is especially important (and this is the one place where af is used as a pair), because a is necessary for getting almost anything done, and f holds the flags which most instructions will invisibly trample.

Naturally, I completely forgot about this the first time around.

The Game Boy has five interrupts, each with a handler at a fixed address very low in ROM. Each handler only has room for eight bytes’ worth of instructions, which is enough to do a very tiny amount of work — or to just jump elsewhere.

A good start is to populate each one with only the reti instruction, which returns as usual and re-enables interrupts. The CPU disables interrupts when it calls an interrupt handler (so they thankfully can’t interrupt themselves), and returning with only ret will leave them disabled.

Naturally, I completely forgot about this the first time around.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
; Interrupt handlers
SECTION "Vblank interrupt", ROM0[$0040]
    ; Fires when the screen finishes drawing the last physical
    ; row of pixels
    reti

SECTION "LCD controller status interrupt", ROM0[$0048]
    ; Fires on a handful of selectable LCD conditions, e.g.
    ; after repainting a specific row on the screen
    reti

SECTION "Timer overflow interrupt", ROM0[$0050]
    ; Fires at a configurable fixed interval
    reti

SECTION "Serial transfer completion interrupt", ROM0[$0058]
    ; Fires when the serial cable is done?
    reti

SECTION "P10-P13 signal low edge interrupt", ROM0[$0060]
    ; Fires when a button is released?
    reti

These will do nothing. I mean, obviously, but they’ll do even less than nothing until I enable them. Interrupts are enabled by the dedicated ei instruction, which enables any interrupts whose corresponding bit is set in the IE register ($ffff).

So… which one do I want?

Game loop

To have a game, I need a game loop. The basic structure of pretty much any loop looks like:

  1. Load stuff.
  2. Check for input.
  3. Update the game state.
  4. Draw the game state.
  5. GOTO 2

(If you’ve never seen a real game loop written out before, LÖVE’s default loop is a good example, though even a huge system like Unity follows the same basic structure.)

The Game Boy seems to introduce a wrinkle here. I don’t actually draw anything myself; rather, the hardware does the drawing, and I tell it what to draw by using the palette registers, OAM, and VRAM.

But in fact, this isn’t too far off from how LÖVE (or Unity) works! All the drawing I do is applied to a buffer, not the screen; once the drawing is complete, the main loop calls present(), which waits until vblank and then draws the buffer to the screen. So what you see on the screen is delayed by up to a frame, and the loop really has an extra “wait for vsync” step at 3½. Or, with a little rearrangement:

  1. Load stuff.
  2. Wait for vblank.
  3. Draw the game state.
  4. Check for input.
  5. Update the game state.
  6. GOTO 2

This is approaching something I can implement! It works out especially well because it does all the drawing as early as possible during vblank. That’s good, because the LCD operation looks something like this:

1
2
3
4
5
6
7
LCD redrawing...
LCD redrawing...
LCD redrawing...
LCD redrawing...
VBLANK
LCD idle
LCD idle

While the LCD is refreshing, I can’t (easily) update anything it might read from. I only have free control over VRAM et al. during a short interval after vblank, so I need to do all my drawing work right then to ensure it happens before the LCD starts refreshing again. Then I’m free to update the world while the LCD is busy.

First, right at the entry point, I enable the vblank interrupt. It’s bit 0 of the IE register, but hardware.inc has me covered.

1
2
3
4
5
main:
    ; Enable interrupts
    ld a, IEF_VBLANK
    ldh [rIE], a
    ei

Next I need to make the handler actually do something. The obvious approach is for the handler to call one iteration of the game loop, but there are a couple problems with that. For one, interrupts are disabled when a handler is called, so I would never get any other interrupts. I could explicitly re-enable interrupts, but that raises a bigger question: what happens if the game lags, and updating the world takes longer than a frame? With this approach, the game loop would interrupt itself and then either return back into itself somewhere and cause untold chaos, or take too long again and eventually overflow the stack. Neither is appealing.

An alternative approach, which I found in gb-template but only truly appreciated after some thought, is for the vblank handler to set a flag and immediately return. The game loop can then wait until the flag is set before each iteration, just like LÖVE does. If an update takes longer than a frame, no problem: the loop will always wait until the next vblank, and the game will simply run more slowly.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
SECTION "Vblank interrupt", ROM0[$0040]
    push hl
    ld hl, vblank_flag
    ld [hl], 1
    pop hl
    reti

...

SECTION "Important twiddles", WRAM0[$C000]
; Reserve a byte in working RAM to use as the vblank flag
vblank_flag:
    db

The handler fits in eight bytes — the linker would yell at me if it didn’t, since another section starts at $0048! — and leaves all the registers in their previous states. As I mentioned before, I originally neglected to preserve registers, and some zany things started to happen as a and f were abruptly altered in the middle of other code. Whoops!

Now the main loop can look like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
main:
    ; ... bunch of setup code ...

vblank_loop:
    ; Main loop: halt, wait for a vblank, then do stuff

    ; The halt instruction stops all CPU activity until the
    ; next interrupt, which saves on battery, or at least on
    ; CPU cycles on an emulator's host system.
    halt
    ; The Game Boy has some obscure hardware bug where the
    ; instruction after a halt is occasionally skipped over,
    ; so every halt should be followed by a nop.  This is so
    ; ubiquitous that rgbasm automatically adds a nop after
    ; every halt, so I don't even really need this here!
    nop

    ; Check to see whether that was a vblank interrupt (since
    ; I might later use one of the other interrupts, all of
    ; which would also cancel the halt).
    ld a, [vblank_flag]
    ; This sets the zero flag iff a is zero
    and a
    jr z, vblank_loop
    ; This always sets a to zero, and is shorter (and thus
    ; faster) than ld a, 0
    xor a, a
    ld [vblank_flag], a

    ; Use DMA to update object attribute memory.
    ; Do this FIRST to ensure that it happens before the screen starts to update again.
    call $FF80

    ; ... update everything ...

    jp vblank_loop

It’s looking all the more convenient that I have my own copy of OAM — I can update it whenever I want during this loop! I might need similar facilities later on for editing VRAM or changing palettes.

Doing something and reading input

I have a loop, but since nothing’s happening, that’s not especially obvious. Input would take a little effort, so I’ll try something simpler first: making Anise move around.

I don’t actually track Anise’s position anywhere right now, except for in the OAM buffer. Good enough. In my main loop, I add:

1
2
3
4
    ld hl, oam_buffer + 1
    ld a, [hl]
    inc a
    ld [hl], a

The second byte in each OAM entry is the x-coordinate, and indeed, this causes Anise’s torso to glide rightwards across the screen at 60ish pixels per second. Eventually the x-coordinate overflows, but that’s fine; it wraps back to zero and moves the sprite back on-screen from the left.

The half-cat is now sliding across the screen

Excellent. I mean, sorry, this is extremely hard to look at, but bear with me a second.

This would be a bit more game-like if I could control it with the buttons, so let’s read from them.

There are eight buttons: up, down, left, right, A, B, start, select. There are also eight bits in a byte. You might suspect that I can simply read an I/O register to get the current state of all eight buttons at once.

Ha, ha! You naïve fool. Of course it’s more convoluted than that. That single byte thing is a pretty good idea, though, so what I’ll do is read the input at the start of the frame and coax it into a byte that I can consult more easily later.

Turns out I pretty much have to do that, because button access is slightly flaky. Even the official manual advises reading the buttons several times to get a reliable result. Yikes.

Here’s how to do it. The buttons are wired in two groups of four: the dpad and everything else. Reading them is thus also done in two groups of four. I need to use the P1 register, which I assume is short for “player 1” and is so named because the people who designed this hardware had also designed the two-player NES?

Bits 5 and 6 of P1 determine which set of four buttons I want to read, and then the lower nybble contains the state of those buttons. Note that each bit is set to 1 if the button is released; I think this is a quirk of how they’re wired, and what I’m doing is extremely direct hardware access. Exciting! (Also very confusing on my first try, where Anise’s movement was inverted.)

The code, which is very similar to an example in the official manual, thus looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
    ; Poll input
    ; The direct hardware access is nonsense and unreliable, so
    ; just read once per frame and stick all the button states
    ; in a byte

    ; Bit 6 means to read the dpad
    ld a, $20
    ldh [rP1], a
    ; But it's unreliable, so do it twice
    ld a, [rP1]
    ld a, [rP1]
    ; This is 'complement', and flips all the bits in a, so now
    ; set bits will mean a button is held down
    cpl
    ; Store the lower four bits in b
    and a, $0f
    ld b, a

    ; Bit 5 means to read the buttons
    ld a, $10
    ldh [rP1], a
    ; Apparently this is even more unreliable??  No, really, the
    ; manual does this: two reads, then six reads
    ld a, [rP1]
    ld a, [rP1]
    ld a, [rP1]
    ld a, [rP1]
    ld a, [rP1]
    ld a, [rP1]
    ; Again, complement and mask off the lower four bits
    cpl
    and a, $0f
    ; b already contains four bits, so I need to shift something
    ; left by four...  but the shift instructions only go one
    ; bit at a time, ugh!  Luckily there's swap, which swaps the
    ; high and low nybbles in any register
    swap a
    ; Combine b's lower nybble with a's high nybble
    or a, b
    ; And finally store it in RAM
    ld [buttons], a

...

SECTION "Important twiddles", WRAM0[$C000]
vblank_flag:
    db
buttons:
    db

Phew. That was a bit of a journey, but now I have the button state as a single byte. To help with reading the buttons, I’ll also define a few constants labeling the individual bits. (There are instructions for reading a particular bit by number, so I don’t need to mask a single bit out.)

1
2
3
4
5
6
7
8
9
; Constants
BUTTON_RIGHT  EQU 0
BUTTON_LEFT   EQU 1
BUTTON_UP     EQU 2
BUTTON_DOWN   EQU 3
BUTTON_A      EQU 4
BUTTON_B      EQU 5
BUTTON_START  EQU 6
BUTTON_SELECT EQU 7

Now to adjust the sprite position based on what directions are held down. Delete the old code and replace it with:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
    ; Set b/c to the y/x coordinates
    ld hl, oam_buffer
    ld b, [hl]
    inc hl
    ld c, [hl]

    ; This sets the z flag to match a particular bit in a
    bit BUTTON_LEFT, a
    ; If z, the bit is zero, so left isn't held down
    jr z, .skip_left
    ; Otherwise, left is held down, so decrement x
    dec c
.skip_left:

    ; The other three directions work the same way
    bit BUTTON_RIGHT, a
    jr z, .skip_right
    inc c
.skip_right:
    bit BUTTON_UP, a
    jr z, .skip_up
    dec b
.skip_up:
    bit BUTTON_DOWN, a
    jr z, .skip_down
    inc b
.skip_down:

    ; Finally, write the new coordinates back to the OAM
    ; buffer, which hl is still pointing into
    ld [hl], c
    dec hl
    ld [hl], b

Miraculously, Anise’s torso now moves around on command!

The half-cat is now moving according to button presses

Neat! But this still looks really, really, incredibly bad.

Aesthetics

It’s time to do something about this artwork.

First things first: I’m really tired of writing out colors by hand, in binary, so let’s fix that. In reality, I did this bit after adding better art, but doing it first is better for everyone.

I think I’ve mentioned before that rgbasm has (very, very rudimentary) support for macros, and this seems like a perfect use case for one. I’d like to be able to write colors out in typical rrggbb hex fashion, so I need to convert a 24-bit color to a 16-bit one.

1
2
3
4
5
6
dcolor: MACRO  ; $rrggbb -> gbc representation
_r = ((\1) & $ff0000) >> 16 >> 3
_g = ((\1) & $00ff00) >> 8  >> 3
_b = ((\1) & $0000ff) >> 0  >> 3
    dw (_r << 0) | (_g << 5) | (_b << 10)
    ENDM

This is going to need a whole paragraph of caveats.

A macro is contained between MACRO and ENDM. The assembler has a curious sort of universal assignment syntax, where even ephemeral constructs like macros are introduced by labels. Macros can take arguments, but they aren’t declared; they’re passed more like arguments to shell scripts, where the first argument is \1 and so forth. (There’s even a SHIFT command for accessing arguments beyond the ninth.) Also, passing strings to a macro is some kind of byzantine nightmare where you have to slap backslashes in just the right places and I will probably avoid doing it altogether if I can at all help it.

Oh, one other caveat: compile-time assignments like I have above must start in the first column. I believe this is because assignments are also labels, and labels have to start in the first column. It’s a bit weird and apparently rgbasm’s lexer is horrifying, but I’ll take it over writing my own assembler and stretching this project out any further.

Anyway, all of that lets me write dcolor $ff0044 somewhere and have it translated at compile time to the appropriate 16-bit value. (I used dcolor to parallel db and friends, but I’m strongly considering using CamelCase exclusively for macros? Guess it depends how heavily I use them.)

With that on hand, I can now doodle some little sprites in Aseprite and copy them in. This part is not especially interesting and involves a lot of squinting at zoomed-in sprites.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
SECTION "Sprites", ROM0
PALETTE_BG0:
    dcolor $80c870  ; light green
    dcolor $48b038  ; darker green
    dcolor $000000  ; unused
    dcolor $000000  ; unused
PALETTE_ANISE:
    dcolor $000000  ; TODO
    dcolor $204048
    dcolor $20b0b0
    dcolor $f8f8f8
GRASS_SPRITE:
    dw `00000000
    dw `00000000
    dw `01000100
    dw `01010100
    dw `00010000
    dw `00000000
    dw `00000000
    dw `00000000
EMPTY_SPRITE:
    dw `00000000
    dw `00000000
    dw `00000000
    dw `00000000
    dw `00000000
    dw `00000000
    dw `00000000
    dw `00000000
ANISE_SPRITE:
    ; ... I'll revisit this momentarily

Gorgeous. You may notice that I put the colors as data instead of inlining them in code, which incidentally makes the code for setting the palette vastly shorter as well:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
    ; Start setting the first color, and advance the internal
    ; pointer on every write
    ld a, %10000000
    ; BCPS = Background Color Palette Specification
    ldh [rBCPS], a

    ld hl, PALETTE_BG0
    REPT 8
    ld a, [hl+]
    ; Same, but Data
    ld [rBCPD], a
    ENDR

Loading sprites into VRAM also becomes a bit less of a mess:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
    ; Load some basic tiles
    ld hl, $8000

    ; Read the 16-byte empty sprite into tile 0
    ld bc, EMPTY_SPRITE
    REPT 16
    ld a, [bc]
    inc bc
    ld [hl+], a
    ENDR

    ; Read the grass sprite into tile 1, which immediately
    ; follows tile 0, so hl is already in the right place
    ld bc, GRASS_SPRITE
    REPT 16
    ld a, [bc]
    inc bc
    ld [hl+], a
    ENDR

Someday I should write an actual copy function, since at the moment, I’m using an alarming amount of space for pointlessly unrolled loops. Maybe later.

You may notice I now have two tiles, whereas before I was relying on filling the entire screen with one tile, tile 0. I want to dot the landscape with tile 1, which means writing a bit more to the actual background grid, which begins at $9800 and has one byte per tile.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
    ; Fill the screen buffer with a pattern of grass tiles,
    ; where every 2x2 block has a single grass at the top left.
    ; Note that the buffer is 32x32 tiles, and it ends at $9c00
    ld hl, $9800
.screen_fill_loop:
    ; Use tile 1 for every other tile in this row.  Note that
    ; REPTed part increments hl /twice/, thus skipping a tile
    ld a, $01
    REPT 16
    ld [hl+], a
    inc hl
    ENDR
    ; Skip an entire row of 32 tiles, which will remain empty.
    ; There is almost certainly a better way to do this, but I
    ; didn't do it.  (Hint: it's ld bc, $20; add hl, bc)
    REPT 32
    inc hl
    ENDR
    ; If we haven't reached $9c00 yet, continue looping
    ld a, h
    cp a, $9C
    jr c, .screen_fill_loop

Sorry for all these big blocks of code, but check out this payoff!

A very simple grassy background

POW! Gorgeous.

And hey, why stop there? With a little more pixel arting against a very reduced palette…

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
SPRITE_ANISE_FRONT_1:
    dw `00000111
    dw `00001222
    dw `00012222
    dw `00121222
    dw `00121122
    dw `00121111
    dw `00121122
    dw `00121312
    dw `00121313
    dw `00012132
    dw `00001211
    dw `00000123
    dw `00100123
    dw `00011133
    dw `00000131
    dw `00000010
SPRITE_ANISE_FRONT_2:
    dw `11100000
    dw `22210000
    dw `22221000
    dw `22212100
    dw `22112100
    dw `11112100
    dw `22112100
    dw `21312100
    dw `31312100
    dw `23121000
    dw `11210000
    dw `32100000
    dw `32100000
    dw `33100000
    dw `13100000
    dw `01000000

Yes, I am having trouble deciding on a naming convention.

This is now a 16×16 sprite, made out of two 8×16 parts. This post has enough code blocks as it is, and the changes to make this work are relatively minor copy/paste work, so the quick version is:

  1. Set the LCDC flag (bit 2, or LCDCF_OBJ16) that makes objects be 8×16. This mode uses pairs of tiles, so an object that uses either tile 0 or 1 will draw both of them, with tile 0 on top of tile 1.
  2. Extend the code that loads object tiles to load four instead.
  3. Define a second sprite that’s 8 pixels to the right of the first one.
  4. Remove the hard-coded object palette, and instead load the PALETTE_ANISE that I sneakily included above. This time the registers are called rOCPS and rOCPD.

Finally, extend the code that moves the sprite to also move the second half:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
    ; Finally, write the new coordinates back to the OAM
    ; buffer, which hl is still pointing into
    ld [hl], c
    dec hl
    ld [hl], b
    ; This bit is new: copy the x-coord into a so I can add 8
    ; to it, then store both coords into the second sprite's
    ; OAM data
    ld a, c
    add a, 8
    ; I could've written this the other way around, but I did
    ; not, I guess because this structure mirrors the above?
    ld hl, oam_buffer + 5
    ld [hl], a
    dec hl
    ld [hl], b

Cross my fingers, and…

A little cat sprite atop the grassy background

Hey hey hey! That finally looks like something!

To be continued

It was a surprisingly long journey, but this brings us more or less up to commit 313a3e, which happens to be the first commit I made a release of! It’s been more than a week, so you can grab it on Patreon or GitHub. I strongly recommend playing it with a release of mGBA prior to 0.7, for… reasons that will become clear next time.

Next time: I’ll take a breather and clean up a few things.

Cheezball Rising: Drawing a sprite

Post Syndicated from Eevee original https://eev.ee/blog/2018/06/21/cheezball-rising-drawing-a-sprite/

This is a series about Star Anise Chronicles: Cheezball Rising, an expansive adventure game about my cat for the Game Boy Color. Follow along as I struggle to make something with this bleeding-edge console!

source codeprebuilt ROMs (a week early for $4) • works best with mGBA

In this issue, I figure out how to draw a sprite. This part was hard.

Previously: figuring out how to put literally anything on the goddamn screen.

Recap

Welcome back! I’ve started cobbling together a Pygments lexer for RGBDS’s assembly flavor, so hopefully the code blocks are more readable, and will become moreso over time.

When I left off last time, I had… um… this.

Vertical stripes of red, green, blue, and white

This is all on the background layer, which I mentioned before is a fixed grid of 8×8 tiles.

For anything that moves around freely, like the player, I need to use the object layer. So that’s an obvious place to go next.

Now, if you remember, I can define tiles by just writing to video RAM, and I define palettes with a goofy system involving writing them one byte at a time to the same magic address. You might expect defining objects to do some third completely different thing, and you’d be right!

Defining an object

Objects are defined in their own little chunk of RAM called OAM, for object attribute memory. They’re also made up of tiles, but each tile can be positioned at an arbitrary point on the screen.

OAM starts at $fe00 and each object takes four bytes — the y-coordinate, the x-coordinate, the tile number, and some flags — for a total of 160 bytes. There are some curiosities, like how the top left of the screen is (8, 10) rather than (0, 0), but I’ll figure out what’s up with that later. (I suppose if zeroes meant the upper left corner, there’d be a whole stack of tile 0 there all the time.)

Here’s the fun part: I can’t write directly to OAM? I guess??? Come to think of it, I don’t think the manual explicitly says I can’t, but it’s strongly implied. Hmm. I’ll look into that. But I didn’t at the time, so I’ll continue under the assumption that the following nonsense is necessary.

Because I “can’t” write directly, I need to use some shenanigans. First, I need something to write! This is an Anise game, so let’s go for Anise.

I’m on my laptop at this point without access to the source code for the LÖVE Anise game I started, so I have to rustle up a screenshot I took.

Cropped screenshot of Star Anise and some critters, all pixel art

Wait a second.

Even on the Game Boy Color, tiles are defined with two bits per pixel. That means an 8×8 tile has a maximum of four colors. For objects, the first color is transparent, so I really have three colors — which is exactly why most Game Boy Color protagonists have a main color, an outline/shadow color, and a highlight color.

Let’s check out that Anise in more detail.

Star Anise at 8×

Hm yes okay that’s more than three colors. I guess I’m going to need to draw some new sprites from scratch, somehow.

In the meantime, I optimistically notice that Star Anise’s body only uses three colors, and it’s 8×7! I could make a tile out of that! I painstakingly copy the pixels into a block of those backticks, which you can kinda see is his body if you squint a bit:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
SECTION "Sprites", ROM0
ANISE_SPRITE:
    dw `00000000
    dw `00001333
    dw `00001323
    dw `10001233
    dw `01001333
    dw `00113332
    dw `00003002
    dw `00003002

The dw notation isn’t an opcode; it tells the assembler to put two literal bytes of data in the final ROM. A word of data. (Each row of a tile is two bytes, remember.)

If you think about this too hard, you start to realize that both the data and code are just bytes, everything is arbitrary, and true meaning is found only in the way we perceive things rather than in the things themselves.

Note I didn’t specify an exact address for this section, so the linker will figure out somewhere to put it and make sure all the labels are right at the end.

Now I load this into tilespace, back in my main code:

1
2
3
4
5
6
7
8
    ; Define an object
    ld hl, $8800
    ld bc, ANISE_SPRITE
    REPT 16
    ld a, [bc]
    ld [hl+], a
    inc bc
    ENDR

This copies 16 bytes, starting from the ANISE_SPRITE label, to $8800.


Why $8800, not $8000? I’m so glad you asked!

There are actually three blocks of tile space, each with enough room for 128 tiles: one at $8000, one at $8800, and one at $9000. Object tiles always use the $8000 block followed by the $8800 block, whereas background tiles can use either $8000 + $8800 or $9000 + $8800. By default, background tiles use $8000 + $8800.

All of which is to say that I got very confused reading the manual (which spends like five pages explaining the above paragraph) and put the object tiles in the wrong place. Whoops. It’s fine; this just ends up being tile 128.

In my partial defense, looking at it now, I see the manual is wrong! Bit 4 of the LCD controller register ($ff40) controls whether the background uses tiles from $8000 + $8800 (1) or $9000 + $8800 (0). The manual says that this register defaults to $83, which has bit 4 off, suggesting that background tiles use $9000 + $8800 (i.e. start at $8800), but disassembly of the boot ROM shows that it actually defaults to $91, which has bit 4 on. Thanks a lot, Nintendo!

That was quite a diversion. Here’s a chart of where the dang tiles live. Note that the block at $8800 is always shared between objects and background tiles. Oh, and on the Game Boy Color, all three blocks are twice as big thanks to the magic of banking. I’ll get to banking… much later.

1
2
3
4
5
                            bit 4 ON (default)  bit 4 OFF
                            ------------------  ---------
$8000   obj tiles 0-127     bg tiles 0-127
$8800   obj tiles 128-255   bg tiles 128-255    bg tiles 128-255
$9000                                           bg tiles 0-127

Hokay. What else? I’m going to need a palette for this, and I don’t want to use that gaudy background palette. Actually, I can’t — the background and object layers have two completely separate sets of palettes.

Writing an object palette is exactly the same as writing a background palette, except with different registers.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
    ; This should look pretty familiar
    ld a, %10000000
    ld [$ff6a], a

    ld bc, %0000000000000000  ; transparent
    ld a, c
    ld [$ff6b], a
    ld a, b
    ld [$ff6b], a
    ld bc, %0010110100100101  ; dark
    ld a, c
    ld [$ff6b], a
    ld a, b
    ld [$ff6b], a
    ld bc, %0100000111001101  ; med
    ld a, c
    ld [$ff6b], a
    ld a, b
    ld [$ff6b], a
    ld bc, %0100001000010001  ; white
    ld a, c
    ld [$ff6b], a
    ld a, b
    ld [$ff6b], a

Riveting!

I wrote out those colors by hand. The original dark color, for example, was #264a59. That uses eight bits per channel, but the Game Boy Color only supports five (a factor of 8 difference), so first I rounded each channel to the nearest 8 and got #284858. Swap the channels to get 58 48 28 and convert to binary (sans the trailing zeroes) to get 01011 01001 00101.

Note to self: probably write a macro or whatever so I can define colors like a goddamn human being. Also why am I not putting the colors in a ROM section too?

Almost there. I still need to write out those four bytes that specify the tile and where it goes. I can’t actually write them to OAM yet, so I need some scratch space in regular RAMworking RAM.

1
2
3
SECTION "OAM Buffer", WRAM0[$C100]
oam_buffer:
    ds 4 * 40

The ds notation is another “data” variant, except it can take a size and reserves space for a whole string of data. Note that I didn’t put any actual data here — this section is in RAM, which only exists while the game is running, so there’d be nowhere to put data.

Also note that I gave an explicit address this time. The buffer has to start at an address ending in 00, for reasons that will become clear momentarily. The space from $c000 to $dfff is available as working RAM, and I chose $c100 for… reasons that will also become clear momentarily.

Now to write four bytes to it at runtime:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
    ; Put an object on the screen
    ld hl, oam_buffer
    ; y-coord
    ld a, 64
    ld [hl+], a
    ; x-coord
    ld [hl+], a
    ; tile index
    ld a, 128
    ld [hl+], a
    ; attributes, including palette, which are all zero
    ld a, %00000000
    ld [hl+], a

(I tried writing directly to OAM on my first attempt. Nothing happened! Very exciting.)

But how to get this into OAM so it’ll actually show on-screen? For that, I need to do a DMA transfer.

DMA

DMA, or direct memory access, is one of those things the Game Boy programming manual seems to think everyone is already familiar with. It refers generally to features that allow some other hardware to access memory, without going through the CPU. In the case of the Game Boy, it’s used to copy data from working RAM to OAM. Only to OAM. It’s very specific.

Performing a DMA transfer is super easy! I write the high byte of the source address to the DMA register ($ff46), and then some magic happens, and 160 bytes from the source address appear in OAM. In other words:

1
2
3
    ld a, $c1       ; copy from $c100
    ld [$ff46], a   ; perform DMA transfer
    ; now $c000 through $c09f have been copied into OAM!

It’s almost too good to be true! And it is. There are some wrinkles.

First, the transfer takes some time, during which I almost certainly don’t want to be doing anything else.

Second, during the transfer, the CPU can only read from “high RAM” — $ff80 and higher. Wait, uh oh.

The usual workaround here is to copy a very short function into high RAM to perform the actual transfer and wait for it to finish, then call that instead of starting a transfer directly. Well, that sounds like a pain, so I break my rule of accounting for every byte and find someone else who’s done it. Conveniently enough, that post is by the author of the small template project I’ve been glancing at.

I end up with something like the following.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
    ; Copy the little DMA routine into high RAM
    ld bc, DMA_BYTECODE
    ld hl, $ff80
    ; DMA routine is 13 bytes long
    REPT 13
    ld a, [bc]
    inc bc
    ld [hl+], a
    ENDR

; ...

SECTION "DMA Bytecode", ROM0
DMA_BYTECODE:
    db $F5, $3E, $C1, $EA, $46, $FF, $3E, $28, $3D, $20, $FD, $F1, $D9

That’s compiled assembly, written inline as bytes. Oh boy. The original code looks like:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
    ; start the transfer, as shown above
    ld a, $c1
    ld [$ff46], a

    ; wait 160 cycles/microseconds, the time it takes for the
    ; transfer to finish; this works because 'dec' is 1 cycle
    ; and 'jr' is 3, for 4 cycles done 40 times
    ld      a, 40
loop:
    dec     a
    jr      nz, loop

    ; return
    ret

Now you can see why I used $c100 for my OAM buffer: because it’s the address this person used.

(Hm, the opcode reference I usually use seems to have all the timings multiplied by a factor of 4 without comment? Odd. The rgbds reference is correct.)

(Also, here’s a fun fact: the stack starts at $fffe and grows backwards. If it grows too big, the very first thing it’ll overwrite is this DMA routine! I bet that’ll have some fun effects.)

At this point I have a thought. (Okay, I had the thought a bit later, but it works better narratively if I have it now.) I’ve already demonstrated that the line between code and data is a bit fuzzy here. So why does this code need to be pre-assembled?

And a similar thought: why is the length hardcoded? Surely, we can do a little better. What if we shuffle things around a bit…

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
SECTION "init", ROM0[$0100]
    nop
    ; Jump to a named label instead of an address
    jp main

SECTION "main", ROM0[$0150]
; DMA copy routine, copied into high RAM at startup.
; Never actually called where it is.
dma_copy:
    ld a, $c1
    ld [$ff46], a
    ld a, 40
.loop:
    dec a
    jr nz, .loop
    ret
dma_copy_end:
    nop

main:
    ; ... all previous code is here now ...

    ; Copy the little DMA routine into high RAM
    ld bc, dma_copy
    ld hl, $ff80
    ; DMA routine is 13 bytes long
    REPT dma_copy_end - dma_copy
    ld a, [bc]
    inc bc
    ld [hl+], a
    ENDR

This is very similar to what I just had, except that the code is left as code, and its length is computed by having another label at the end — so I’m free to edit it later if I want to. It all ends up as bytes in the ROM, so the code ends up exactly the same as writing out the bytes with db. Come to think of it, I don’t even need to hardcode the $c1 there; I could replace it with oam_buffer >> 8 and avoid repeating myself.

(I put the code at $0150 because rgbasm is very picky about subtracting labels, and will only do it if they both have fixed positions. These two labels would be the same distance apart no matter where I put the section, but I guess rgbasm isn’t smart enough to realize that.)

I’m actually surprised that the author of the above post didn’t think to do this? Maybe it’s dirty even by assembly standards.

Timing, vblank, and some cool trickery

Okay, so, as I was writing that last section, I got really curious about whether and when I’m actually allowed to write to OAM. Or tile RAM, for that matter.

I found/consulted the Game Boy dev wiki, and the rules match what’s in the manual, albeit with a chart that makes things a little more clear.

My understanding is as follows. The LCD draws the screen one row of pixels at a time, and each row has the following steps:

  1. Look through OAM to see if any sprites are on this row. OAM is inaccessible to the CPU.

  2. Draw the row. OAM, VRAM, and palettes are all inaccessible.

  3. Finish the row and continue on to the beginning of the next row. This takes a nonzero amount of time, called the horizontal blanking period, during which the CPU can access everything freely.

Once the LCD reaches the bottom, it continues to “draw” a number of faux rows below the bottom of the visible screen (vertical blanking), and the CPU can again do whatever it wants. Eventually it returns to the top-left corner to draw again, concluding a single frame. The entire process happens 59.7 times per second.

There’s one exception: DMA transfers can happen any time, but the LCD will simply not draw sprites during the transfer.

So I probably shouldn’t be writing to tiles and palettes willy-nilly. I suspect I got away with it because it happened in that first OAM-searching stage… and/or because I did it on emulators which are a bit more flexible than the original hardware.

In fact…

Same screenshot as above, but the first row of pixels is corrupt

I took this screenshot by loading the ROM I have so far, pausing it, resetting it, and then advancing a single frame. This is the very first frame my game shows. If you look closely at the first row of pixels, you can see they’re actually corrupt — they’re being drawn before I’ve set up the palette! You can even see each palette entry taking effect along the row.

This is very cool. It also means my current code would not work at all on actual hardware. I should probably just turn the screen off while I’m doing setup like this.

It’s interesting that only OAM gets a special workaround in the form of a DMA transfer — I imagine because sprites move around much more often than the tileset changes — but having the LCD stop drawing sprites in the meantime is quite a limitation. Surely, you’d only want to do a DMA transfer during vblank anyway? It is much faster than copying by hand, so I’ll still take it.

All of this is to say: I’m gonna need to care about vblanks.


Incidentally, the presence of hblank is very cool and can be used for a number of neat effects, especially when combined with the Game Boy’s ability to call back into user code when the LCD reaches a specific row:

  • The GBC Zelda games use it for map scrolling. The status bar at the top is in one of the two background maps, and as soon as that finishes drawing, the game switches to the other one, which contains the world.

  • Those same games also use it for a horizontal wavy effect, both when warping around and when underwater — all they need to do is change the background layer’s x offset during each hblank!

  • The wiki points out that OAM could be written to in the middle of a screen update, thus bypassing the 40-object restriction: draw 40 objects on the top half of the screen, swap out OAM midway, and then the LCD will draw a different 40 on the bottom half!

  • I imagine you could also change palettes midway through a redraw and exceed the usual limit of 56 colors on screen at a time! No telling whether this sort of trick would work on an emulator, though.

I am very excited at the prospects here.

I’m also slightly terrified. I have a fixed amount of time between frames, and with the LCD as separate hardware, there’s no such thing as a slow frame. If I don’t finish, things go bad. And that time is measured in instructions — an ld always takes the same number of cycles! There’s no faster computer or reducing GC pressure. There’s just me. Yikes.

Back to drawing a sprite

I haven’t had a single new screenshot this entire post! This is ridiculous. All I want is to draw a thing to the screen.

I have some data in my OAM buffer. I have DMA set up. All I should need to do now is start a transfer.

1
    call $ff80

And… nothing. mGBA’s memory viewer confirms everything’s in the right place, but nothing’s on the screen.

Whoops! Remember that LCD controller register, and how it defaults to $91? Well, bit 1 is whether to show objects at all, and it defaults to off. So let’s fix that.

1
2
    ld a, %10010011  ; $91 plus bit 2
    ld [$ff40], a
The same gaudy background, but now with a partial Anise sprite on top

SUCCESS!

It doesn’t look like much, but it took a lot of flailing to get here, and I was overjoyed when I first saw it. The rest should be a breeze! Right?

To be continued

That doesn’t even get us all the way through commit 1b17c7, but this is already more than enough.

Next time: input, and moderately less eye-searing art!

Cheezball Rising: A new Game Boy Color game

Post Syndicated from Eevee original https://eev.ee/blog/2018/06/19/cheezball-rising-a-new-game-boy-color-game/

This is a series about Star Anise Chronicles: Cheezball Rising, an expansive adventure game about my cat for the Game Boy Color. Follow along as I struggle to make something with this bleeding-edge console!

source codeprebuilt ROMs (a week early for $4) • works best with mGBA

In this issue, I figure out how to put literally anything on the goddamn screen, then add a splash of color.

The plan

I’m making a Game Boy Color game!

I have no— okay, not much idea what I’m doing, so I’m going to document my progress as I try to forge a 90s handheld game out of nothing.

I do usually try to keep tech stuff accessible, but this is going to get so arcane that that might be a fool’s errand. Think of this as less of an extended tutorial, more of a long-form Twitter.

Also, I’ll be posting regular builds on Patreon for $4 supporters, which will be available a week later for everyone else. I imagine they’ll generally stay in lockstep with the posts, unless I fall behind on the writing part. But when has that ever happened?

Your very own gamedev legend is about to unfold! A world of dreams and adventures with gbz80 assembly awaits! Let’s go!

Prerequisites

First things first. I have a teeny bit of experience with Game Boy hacking, so I know I need:

  • An emulator. I have no way to run arbitrary code on an actual Game Boy Color, after all. I like mGBA, which strives for accuracy and has some debug tools built in.

    There’s already a serious pitfall here: emulators are generally designed to run games that would work correctly on the actual hardware, but they won’t necessarily reject games that wouldn’t work on actual hardware. In other words, something that works in an emulator might still not work on a real GBC. I would of course prefer that this game work on the actual console it’s built for, but I’ll worry about that later.

  • An assembler, which can build Game Boy assembly code into a ROM. I pretty much wrote one of these myself already for the Pokémon shenanigans, but let’s go with something a little more robust here. I’m using RGBDS, which has a couple nice features like macros and a separate linking step. It compiles super easily, too.

    I also hunted down a vim syntax file, uh, somewhere. I can’t remember which one it was now, and it’s kind of glitchy anyway.

  • Some documentation. I don’t know exactly how this surfaced, but the actual official Game Boy programming manual is on archive.org. It glosses over some things and assumes some existing low-level knowledge, but for the most part it’s a very solid reference.

For everything else, there’s Google, and also the curated awesome-gbdev list of resources.

That list includes several skeleton projects for getting started, but I’m not going to use them. I want to be able to account for every byte of whatever I create. I will, however, refer to them if I get stuck early on. (Spoilers: I get stuck early on.)

And that’s it! The rest is up to me.

Making nothing from nothing

Might as well start with a Makefile. The rgbds root documentation leads me to the following incantation:

1
2
3
4
all:
        rgbasm -o main.o main.rgbasm
        rgblink -o gamegirl.gb main.o
        rgbfix -v -p 0 gamegirl.gb

(I, uh, named this project “gamegirl” before I figured out what it was going to be. It’s a sort of witticism, you see.)

This works basically like every C compiler under the sun, as you might expect: every source file compiles to an object file, then a linker bundles all the object files into a ROM. If I only change one source file, I only have to rebuild one object file.

Of course, this Makefile is terrible garbage and will rebuild the entire project unconditionally every time, but at the moment that takes a fraction of a second so I don’t care.

The extra rgbfix step is new, though — it adds the Nintendo logo (the one you see when you start up a Game Boy) to the header at the beginning of the ROM. Without this, the console will assume the cartridge is dirty or missing or otherwise unreadable, and will refuse to do anything at all. (I could also bake the logo into the source itself, but given that it’s just a fixed block of bytes and rgbfix is bundled with the assembler, I see no reason to bother with that.)

All I need now is a source file, main.rgbasm, which I populate with:

1

Nothing! I don’t know what I expect from this, but I’m curious to see what comes out. And what comes out is a working ROM!

A completely blank screen

Maybe “working” is a strong choice of word, given that it doesn’t actually do anything.

Doing something

It would be fantastic to put something on the screen. This turned out to be harder than expected.

First attempt. I know that the Game Boy starts running code at $0150, immediately after the end of the header. So I’ll put some code there.

A brief Game Boy graphics primer: there are two layers, the background and objects. (There’s also a third layer, the window, which I don’t entirely understand yet.) The background is a grid of 8×8 tiles, two bits per pixel, for a total of four shades of gray. Objects can move around freely, but they lose color 0 to transparency, so they can only use three colors.

There are lots more interesting details and restrictions, which I will think about more later.

Drawing objects is complicated, and all I want to do right now is get something. I’m pretty sure the background defaults to showing all tile 0, so I’ll try replacing tile 0 with a gradient and see what happens.

Tiles are 8×8 and two bits per pixel, which means each row takes two bytes, and the whole tile is 16 bytes. Tiles are defined in one big contiguous block starting at $8000 — or, maybe $8800, sometimes — so all I need to do is:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
SECTION "main", ROM0[$0150]
    ld hl, $8000
    ld a, %00011011
    REPT 16
    ld [hl+], a
    ENDR

_halt:
    ; Do nothing, forever
    halt
    nop
    jr _halt

If you are not familiar with assembly, this series is going to be a wild ride. But here’s a very very brief primer.

Assembly language — really, an assembly language — is little more than a set of human-readable names for the primitive operations a CPU knows how to do. And those operations, by and large, consist of moving bytes around. The names tend to be very short, because you end up typing them a lot.

Most of the work is done in registers, which are a handful of spaces for storing bytes right on the CPU. At this level, RAM is relatively slow — it’s further away, outside the chip — so you want to do as much work as possible in registers. Indeed, most operations can only be done on registers, so there’s a lot of fetching stuff from RAM and operating on it and then putting it back in RAM.

The Game Boy CPU, a modified Z80, has eight byte-sized registers. They’re often referred to in pairs, because they can be paired up to make a 16-bit values (giving you access to a full 64KB address space). And they are: af, bc, de, hl.

The af pair is special. The f register is used for flags, such as whether the last instruction caused an overflow, so it’s not generally touched directly. The a register is called the accumulator and is most commonly used for math operations — in fact, a lot of math operations can only be done on a. The hl register is most often used for addresses, and there are a couple instructions specific to hl that are convenient for memory access. (The h and l even refer to the high and low byte of an address.) The other two pairs aren’t especially noteworthy.

Also! Not every address is actually RAM; the address space ($0000 through $ffff) is carved into several distinct areas, which we will see as I go along. $8000 is the beginning of display RAM, which the screen reads from asynchronously. Also, a lot of addresses above $ff00 (also called “registers”) are special and control hardware in some way, or even perform some action when written to.

With that in mind, here’s the above code with explanatory comments:

TODO need to change this to write a single byte

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
; This is a directive for the assembler to put the following
; code at $0150 in the final ROM.
SECTION "main", ROM0[$0150]
    ; Put the hex value $8000 into registers hl.  Really, that
    ; means put $80 into h and $00 into l.
    ld hl, $8000

    ; Put this binary value into registers a.
    ; It's just 0 1 2 3, a color gradient.
    ld a, %00011011

    ; This is actually a macro this particular assembler
    ; understands, which will repeat the following code 16
    ; times, exactly as if I'd copy-pasted it.
    REPT 16

    ; The brackets (sometimes written as parens) mean to use hl
    ; as a position in RAM, rather than operating on hl itself.
    ; So this copies a into the position in RAM given by
    ; hl (initially $8000), and the + adds 1 to hl afterwards.
    ; This is one reason hl is nice for storing addresses: the +
    ; variant is handy for writing a sequence of bytes to RAM,
    ; and it only exists for hl.
    ld [hl+], a

    ; End the REPT block
    ENDR

; This is a label, used to refer to some position in the code.
; It only exists in the source file.
_halt:
    ; Stop all CPU activity until there's an interrupt.  I
    ; haven't turned any interrupts on, so this stops forever.
    halt

    ; The Game Boy hardware has a bug where, under rare and
    ; unspecified conditions, the instruction after a halt will
    ; be skipped.  So every halt should be followed by a nop,
    ; "no operation", which does nothing.
    nop

    ; This jumps back up to the label.  It's short for "jump
    ; relative", and will end up as an instruction saying
    ; something like "jump backwards five bytes", or however far
    ; back _halt is.  (Different instructions can be different
    ; lengths.)
    jr _halt

Okay! Glad you’re all caught up. The rgbds documentation includes a list of all the available operations (as well as assembler syntax), and once you get used to the short names, I also like this very compact chart of all the instructions and how they compile to machine code. (Note that that chart spells [hl+] as (HLI), for “increment” — the human-readable names are somewhat arbitrary and can sometimes vary between assemblers.)

Now, let’s see what this does!


A completely blank screen, still

Wow! It’s… still nothing. Hang on.

If I open the debugger and hit Break, I find out that the CPU is at address $0120 — before my code — and is on an instruction DD. What’s DD? Well, according to this convenient chart, it’s… nothing. That’s not an instruction.

Hmm.

Problem solving

Maybe it’s time to look at one of those skeleton projects after all. I crack open the smallest one, gb-template, and it seems to be doing the same thing: its code istarts at $0150.

It takes me a bit to realize my mistake here. Practically every Game Boy game starts its code at $0150, but that’s not what the actual hardware specifies. The real start point is $0100, which is immediately before the header! There are only four bytes before the header, just enough for… a jump instruction.

Okay! No problem.

1
2
3
SECTION "entry point", ROM0[$0100]
    nop
    jp $0150

Why the nop? I have no idea, but all of these boilerplate projects do it.

Black screen with repeating columns of white

Uhh.

Well, that’s weird. Not only is the result black and white when I definitely used all four shades, but the whites aren’t even next to each other. (I also had a strange effect where the screen reverted to all white after a few seconds, but can’t reproduce it now; it was fixed by the same steps, though, so it may have been a quirk of a particular mGBA build.)

I’ll save you my head-scratching. I made two mistakes here. Arguably, three!

First: believe it or not, I have to specify the palette. Even in original uncolored Game Boy mode! I can see how that’s nice for doing simple fade effects or flashing colors, but I didn’t suspect it would be necessary. The monochrome palette lives at $ff47 (one of those special high addresses), so I do this before anything else:

1
2
    ld a, %11100100         ; 3 2 1 0
    ld [$ff47], a

I should really give names to some of these special addresses, but for now I’m more interested in something that works than something that’s nice to read.

Second: I specified the colors wrong. I assumed that eight pixels would fit into two bytes as AaBbCcDd EeFfGgHh, perhaps with some rearrangement, but a closer look at Nintendo’s manual reveals that they need to be ABCDEFGH abcdefgh, with the two bits for each pixel split across each byte! Wild.

Handily, rgbds has syntax for writing out pixel values directly: a backtick followed by eight of 0, 1, 2, and 3. I just have to change my code a bit to write two bytes, eight times each. By putting a 16-bit value in a register pair like bc, I can read its high and low bytes out individually via the b and c registers.

1
2
3
4
5
6
7
8
    ld hl, $8000
    ld bc, `00112233
    REPT 8
    ld a, b
    ld [hl+], a
    ld a, c
    ld [hl+], a
    ENDR

Third: strictly speaking, I don’t think I should be writing to $8000 while the screen is on, because the screen may be trying to read from it at the same time. It does happen to work in this emulator, but I have no idea whether it would work on actual hardware. I’m not going to worry too much about this test code; most likely, tile loading will happen all in one place in the real game, and I can figure out any issues then.

This is one of those places where the manual is oddly vague. It dedicates two whole pages to diagrams of how sprites are drawn when they overlap, yet when I can write to display RAM is left implicit.

Well, whatever. It works on my machine.

Stripes of varying shades of gray

Success! I made a thing for the Game Boy.

Ah, but what I wanted was a thing for the Game Boy Color. That shouldn’t be too much harder.

Now in Technicolor

First I update my Makefile to pass the -C flag to rgbfix. That tells it to set a flag in the ROM header to indicate that this game is only intended for the Game Boy Color, and won’t work on the original Game Boy. (In order to pass Nintendo certification, I’ll need an error screen when the game is run on a non-Color Game Boy, but that can come later. Also, I don’t actually know how to do that.)

Oh, and I’ll change the file extension from .gb to .gbc. And while I’m in here, I might as well repeat myself slightly less in this bad, bad Makefile.

1
2
3
4
5
6
7
8
TARGET := gamegirl.gbc

all: $(TARGET)

$(TARGET):
        rgbasm -o main.o main.rgbasm
        rgblink -o $(TARGET) main.o
        rgbfix -C -v -p 0 $(TARGET)

I think := is the one I want, right? Christ, who can remember how this syntax works.

Next I need to define a palette. Again, everything defaults to palette zero, so I’ll update that and not have to worry about specifying a palette for every tile.

This part is a bit weird. Unlike tiles, there’s not a block of addresses somewhere that contains all the palettes. Instead, I have to write the palette to a single address one byte at a time, and the CPU will put it… um… somewhere.

(I think this is because the entire address space was already carved up for the original Game Boy, and they just didn’t have room to expose palettes, but they still had a few spare high addresses they could use for new registers.)

Two registers are involved here. The first, $ff68, specifies which palette I’m writing to. It has a bunch of parts, but since I’m writing to the first color of palette zero, I can leave it all zeroes. The one exception is the high bit, which I’ll explain in just a moment.

1
2
    ld a, %10000000
    ld [$ff68], a

The other, $ff69, does the actual writing. Each color in a palette is two bytes, and a palette contains four colors, so I need to write eight bytes to this same address. The high bit in $ff68 is helpful here: it means that every time I write to $ff69, it should increment its internal position by one. This is kind of like the [hl+] I used above: after every write, the address increases, so I can just write all the data in sequence.

But first I need some colors! Game Boy Color colors are RGB555, which means each color is five bits (0–31) and a full color fits in two bytes: 0bbbbbgg gggrrrrr.

(I got this backwards initially and thought the left bits were red and the right bits were blue.)

Thus, I present, palette loading by hand. Like before, I put the 16-bit color in bc and then write out the contents of b and c. (Before, the backtick syntax put the bytes in the right order; colors are little-endian, hence why I write c before b.)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
    ld bc, %0111110000000000  ; blue
    ld a, c
    ld [$ff69], a
    ld a, b
    ld [$ff69], a
    ld bc, %0000001111100000  ; green
    ld a, c
    ld [$ff69], a
    ld a, b
    ld [$ff69], a
    ld bc, %0000000000011111  ; red
    ld a, c
    ld [$ff69], a
    ld a, b
    ld [$ff69], a
    ld bc, %0111111111111111  ; white
    ld a, c
    ld [$ff69], a
    ld a, b
    ld [$ff69], a

Rebuild, and:

Same as before, but now the stripes are colored

What a glorious eyesore!

To be continued

That brings us up to commit 212344 and works as a good stopping point.

Next time: sprites! Maybe even some real art?

AWS Online Tech Talks – June 2018

Post Syndicated from Devin Watson original https://aws.amazon.com/blogs/aws/aws-online-tech-talks-june-2018/

AWS Online Tech Talks – June 2018

Join us this month to learn about AWS services and solutions. New this month, we have a fireside chat with the GM of Amazon WorkSpaces and our 2nd episode of the “How to re:Invent” series. We’ll also cover best practices, deep dives, use cases and more! Join us and register today!

Note – All sessions are free and in Pacific Time.

Tech talks featured this month:

 

Analytics & Big Data

June 18, 2018 | 11:00 AM – 11:45 AM PTGet Started with Real-Time Streaming Data in Under 5 Minutes – Learn how to use Amazon Kinesis to capture, store, and analyze streaming data in real-time including IoT device data, VPC flow logs, and clickstream data.
June 20, 2018 | 11:00 AM – 11:45 AM PT – Insights For Everyone – Deploying Data across your Organization – Learn how to deploy data at scale using AWS Analytics and QuickSight’s new reader role and usage based pricing.

 

AWS re:Invent
June 13, 2018 | 05:00 PM – 05:30 PM PTEpisode 2: AWS re:Invent Breakout Content Secret Sauce – Hear from one of our own AWS content experts as we dive deep into the re:Invent content strategy and how we maintain a high bar.
Compute

June 25, 2018 | 01:00 PM – 01:45 PM PTAccelerating Containerized Workloads with Amazon EC2 Spot Instances – Learn how to efficiently deploy containerized workloads and easily manage clusters at any scale at a fraction of the cost with Spot Instances.

June 26, 2018 | 01:00 PM – 01:45 PM PTEnsuring Your Windows Server Workloads Are Well-Architected – Get the benefits, best practices and tools on running your Microsoft Workloads on AWS leveraging a well-architected approach.

 

Containers
June 25, 2018 | 09:00 AM – 09:45 AM PTRunning Kubernetes on AWS – Learn about the basics of running Kubernetes on AWS including how setup masters, networking, security, and add auto-scaling to your cluster.

 

Databases

June 18, 2018 | 01:00 PM – 01:45 PM PTOracle to Amazon Aurora Migration, Step by Step – Learn how to migrate your Oracle database to Amazon Aurora.
DevOps

June 20, 2018 | 09:00 AM – 09:45 AM PTSet Up a CI/CD Pipeline for Deploying Containers Using the AWS Developer Tools – Learn how to set up a CI/CD pipeline for deploying containers using the AWS Developer Tools.

 

Enterprise & Hybrid
June 18, 2018 | 09:00 AM – 09:45 AM PTDe-risking Enterprise Migration with AWS Managed Services – Learn how enterprise customers are de-risking cloud adoption with AWS Managed Services.

June 19, 2018 | 11:00 AM – 11:45 AM PTLaunch AWS Faster using Automated Landing Zones – Learn how the AWS Landing Zone can automate the set up of best practice baselines when setting up new

 

AWS Environments

June 21, 2018 | 11:00 AM – 11:45 AM PTLeading Your Team Through a Cloud Transformation – Learn how you can help lead your organization through a cloud transformation.

June 21, 2018 | 01:00 PM – 01:45 PM PTEnabling New Retail Customer Experiences with Big Data – Learn how AWS can help retailers realize actual value from their big data and deliver on differentiated retail customer experiences.

June 28, 2018 | 01:00 PM – 01:45 PM PTFireside Chat: End User Collaboration on AWS – Learn how End User Compute services can help you deliver access to desktops and applications anywhere, anytime, using any device.
IoT

June 27, 2018 | 11:00 AM – 11:45 AM PTAWS IoT in the Connected Home – Learn how to use AWS IoT to build innovative Connected Home products.

 

Machine Learning

June 19, 2018 | 09:00 AM – 09:45 AM PTIntegrating Amazon SageMaker into your Enterprise – Learn how to integrate Amazon SageMaker and other AWS Services within an Enterprise environment.

June 21, 2018 | 09:00 AM – 09:45 AM PTBuilding Text Analytics Applications on AWS using Amazon Comprehend – Learn how you can unlock the value of your unstructured data with NLP-based text analytics.

 

Management Tools

June 20, 2018 | 01:00 PM – 01:45 PM PTOptimizing Application Performance and Costs with Auto Scaling – Learn how selecting the right scaling option can help optimize application performance and costs.

 

Mobile
June 25, 2018 | 11:00 AM – 11:45 AM PTDrive User Engagement with Amazon Pinpoint – Learn how Amazon Pinpoint simplifies and streamlines effective user engagement.

 

Security, Identity & Compliance

June 26, 2018 | 09:00 AM – 09:45 AM PTUnderstanding AWS Secrets Manager – Learn how AWS Secrets Manager helps you rotate and manage access to secrets centrally.
June 28, 2018 | 09:00 AM – 09:45 AM PTUsing Amazon Inspector to Discover Potential Security Issues – See how Amazon Inspector can be used to discover security issues of your instances.

 

Serverless

June 19, 2018 | 01:00 PM – 01:45 PM PTProductionize Serverless Application Building and Deployments with AWS SAM – Learn expert tips and techniques for building and deploying serverless applications at scale with AWS SAM.

 

Storage

June 26, 2018 | 11:00 AM – 11:45 AM PTDeep Dive: Hybrid Cloud Storage with AWS Storage Gateway – Learn how you can reduce your on-premises infrastructure by using the AWS Storage Gateway to connecting your applications to the scalable and reliable AWS storage services.
June 27, 2018 | 01:00 PM – 01:45 PM PTChanging the Game: Extending Compute Capabilities to the Edge – Discover how to change the game for IIoT and edge analytics applications with AWS Snowball Edge plus enhanced Compute instances.
June 28, 2018 | 11:00 AM – 11:45 AM PTBig Data and Analytics Workloads on Amazon EFS – Get best practices and deployment advice for running big data and analytics workloads on Amazon EFS.

Storing Encrypted Credentials In Git

Post Syndicated from Bozho original https://techblog.bozho.net/storing-encrypted-credentials-in-git/

We all know that we should not commit any passwords or keys to the repo with our code (no matter if public or private). Yet, thousands of production passwords can be found on GitHub (and probably thousands more in internal company repositories). Some have tried to fix that by removing the passwords (once they learned it’s not a good idea to store them publicly), but passwords have remained in the git history.

Knowing what not to do is the first and very important step. But how do we store production credentials. Database credentials, system secrets (e.g. for HMACs), access keys for 3rd party services like payment providers or social networks. There doesn’t seem to be an agreed upon solution.

I’ve previously argued with the 12-factor app recommendation to use environment variables – if you have a few that might be okay, but when the number of variables grow (as in any real application), it becomes impractical. And you can set environment variables via a bash script, but you’d have to store it somewhere. And in fact, even separate environment variables should be stored somewhere.

This somewhere could be a local directory (risky), a shared storage, e.g. FTP or S3 bucket with limited access, or a separate git repository. I think I prefer the git repository as it allows versioning (Note: S3 also does, but is provider-specific). So you can store all your environment-specific properties files with all their credentials and environment-specific configurations in a git repo with limited access (only Ops people). And that’s not bad, as long as it’s not the same repo as the source code.

Such a repo would look like this:

project
└─── production
|   |   application.properites
|   |   keystore.jks
└─── staging
|   |   application.properites
|   |   keystore.jks
└─── on-premise-client1
|   |   application.properites
|   |   keystore.jks
└─── on-premise-client2
|   |   application.properites
|   |   keystore.jks

Since many companies are using GitHub or BitBucket for their repositories, storing production credentials on a public provider may still be risky. That’s why it’s a good idea to encrypt the files in the repository. A good way to do it is via git-crypt. It is “transparent” encryption because it supports diff and encryption and decryption on the fly. Once you set it up, you continue working with the repo as if it’s not encrypted. There’s even a fork that works on Windows.

You simply run git-crypt init (after you’ve put the git-crypt binary on your OS Path), which generates a key. Then you specify your .gitattributes, e.g. like that:

secretfile filter=git-crypt diff=git-crypt
*.key filter=git-crypt diff=git-crypt
*.properties filter=git-crypt diff=git-crypt
*.jks filter=git-crypt diff=git-crypt

And you’re done. Well, almost. If this is a fresh repo, everything is good. If it is an existing repo, you’d have to clean up your history which contains the unencrypted files. Following these steps will get you there, with one addition – before calling git commit, you should call git-crypt status -f so that the existing files are actually encrypted.

You’re almost done. We should somehow share and backup the keys. For the sharing part, it’s not a big issue to have a team of 2-3 Ops people share the same key, but you could also use the GPG option of git-crypt (as documented in the README). What’s left is to backup your secret key (that’s generated in the .git/git-crypt directory). You can store it (password-protected) in some other storage, be it a company shared folder, Dropbox/Google Drive, or even your email. Just make sure your computer is not the only place where it’s present and that it’s protected. I don’t think key rotation is necessary, but you can devise some rotation procedure.

git-crypt authors claim to shine when it comes to encrypting just a few files in an otherwise public repo. And recommend looking at git-remote-gcrypt. But as often there are non-sensitive parts of environment-specific configurations, you may not want to encrypt everything. And I think it’s perfectly fine to use git-crypt even in a separate repo scenario. And even though encryption is an okay approach to protect credentials in your source code repo, it’s still not necessarily a good idea to have the environment configurations in the same repo. Especially given that different people/teams manage these credentials. Even in small companies, maybe not all members have production access.

The outstanding questions in this case is – how do you sync the properties with code changes. Sometimes the code adds new properties that should be reflected in the environment configurations. There are two scenarios here – first, properties that could vary across environments, but can have default values (e.g. scheduled job periods), and second, properties that require explicit configuration (e.g. database credentials). The former can have the default values bundled in the code repo and therefore in the release artifact, allowing external files to override them. The latter should be announced to the people who do the deployment so that they can set the proper values.

The whole process of having versioned environment-speific configurations is actually quite simple and logical, even with the encryption added to the picture. And I think it’s a good security practice we should try to follow.

The post Storing Encrypted Credentials In Git appeared first on Bozho's tech blog.

Some quick thoughts on the public discussion regarding facial recognition and Amazon Rekognition this past week

Post Syndicated from Dr. Matt Wood original https://aws.amazon.com/blogs/aws/some-quick-thoughts-on-the-public-discussion-regarding-facial-recognition-and-amazon-rekognition-this-past-week/

We have seen a lot of discussion this past week about the role of Amazon Rekognition in facial recognition, surveillance, and civil liberties, and we wanted to share some thoughts.

Amazon Rekognition is a service we announced in 2016. It makes use of new technologies – such as deep learning – and puts them in the hands of developers in an easy-to-use, low-cost way. Since then, we have seen customers use the image and video analysis capabilities of Amazon Rekognition in ways that materially benefit both society (e.g. preventing human trafficking, inhibiting child exploitation, reuniting missing children with their families, and building educational apps for children), and organizations (enhancing security through multi-factor authentication, finding images more easily, or preventing package theft). Amazon Web Services (AWS) is not the only provider of services like these, and we remain excited about how image and video analysis can be a driver for good in the world, including in the public sector and law enforcement.

There have always been and will always be risks with new technology capabilities. Each organization choosing to employ technology must act responsibly or risk legal penalties and public condemnation. AWS takes its responsibilities seriously. But we believe it is the wrong approach to impose a ban on promising new technologies because they might be used by bad actors for nefarious purposes in the future. The world would be a very different place if we had restricted people from buying computers because it was possible to use that computer to do harm. The same can be said of thousands of technologies upon which we all rely each day. Through responsible use, the benefits have far outweighed the risks.

Customers are off to a great start with Amazon Rekognition; the evidence of the positive impact this new technology can provide is strong (and growing by the week), and we’re excited to continue to support our customers in its responsible use.

-Dr. Matt Wood, general manager of artificial intelligence at AWS

Hiring a Director of Sales

Post Syndicated from Yev original https://www.backblaze.com/blog/hiring-a-director-of-sales/

Backblaze is hiring a Director of Sales. This is a critical role for Backblaze as we continue to grow the team. We need a strong leader who has experience in scaling a sales team and who has an excellent track record for exceeding goals by selling Software as a Service (SaaS) solutions. In addition, this leader will need to be highly motivated, as well as able to create and develop a highly-motivated, success oriented sales team that has fun and enjoys what they do.

The History of Backblaze from our CEO
In 2007, after a friend’s computer crash caused her some suffering, we realized that with every photo, video, song, and document going digital, everyone would eventually lose all of their information. Five of us quit our jobs to start a company with the goal of making it easy for people to back up their data.

Like many startups, for a while we worked out of a co-founder’s one-bedroom apartment. Unlike most startups, we made an explicit agreement not to raise funding during the first year. We would then touch base every six months and decide whether to raise or not. We wanted to focus on building the company and the product, not on pitching and slide decks. And critically, we wanted to build a culture that understood money comes from customers, not the magical VC giving tree. Over the course of 5 years we built a profitable, multi-million dollar revenue business — and only then did we raise a VC round.

Fast forward 10 years later and our world looks quite different. You’ll have some fantastic assets to work with:

  • A brand millions recognize for openness, ease-of-use, and affordability.
  • A computer backup service that stores over 500 petabytes of data, has recovered over 30 billion files for hundreds of thousands of paying customers — most of whom self-identify as being the people that find and recommend technology products to their friends.
  • Our B2 service that provides the lowest cost cloud storage on the planet at 1/4th the price Amazon, Google or Microsoft charges. While being a newer product on the market, it already has over 100,000 IT and developers signed up as well as an ecosystem building up around it.
  • A growing, profitable and cash-flow positive company.
  • And last, but most definitely not least: a great sales team.

You might be saying, “sounds like you’ve got this under control — why do you need me?” Don’t be misled. We need you. Here’s why:

  • We have a great team, but we are in the process of expanding and we need to develop a structure that will easily scale and provide the most success to drive revenue.
  • We just launched our outbound sales efforts and we need someone to help develop that into a fully successful program that’s building a strong pipeline and closing business.
  • We need someone to work with the marketing department and figure out how to generate more inbound opportunities that the sales team can follow up on and close.
  • We need someone who will work closely in developing the skills of our current sales team and build a path for career growth and advancement.
  • We want someone to manage our Customer Success program.

So that’s a bit about us. What are we looking for in you?

Experience: As a sales leader, you will strategically build and drive the territory’s sales pipeline by assembling and leading a skilled team of sales professionals. This leader should be familiar with generating, developing and closing software subscription (SaaS) opportunities. We are looking for a self-starter who can manage a team and make an immediate impact of selling our Backup and Cloud Storage solutions. In this role, the sales leader will work closely with the VP of Sales, marketing staff, and service staff to develop and implement specific strategic plans to achieve and exceed revenue targets, including new business acquisition as well as build out our customer success program.

Leadership: We have an experienced team who’s brought us to where we are today. You need to have the people and management skills to get them excited about working with you. You need to be a strong leader and compassionate about developing and supporting your team.

Data driven and creative: The data has to show something makes sense before we scale it up. However, without creativity, it’s easy to say “the data shows it’s impossible” or to find a local maximum. Whether it’s deciding how to scale the team, figuring out what our outbound sales efforts should look like or putting a plan in place to develop the team for career growth, we’ve seen a bit of creativity get us places a few extra dollars couldn’t.

Jive with our culture: Strong leaders affect culture and the person we hire for this role may well shape, not only fit into, ours. But to shape the culture you have to be accepted by the organism, which means a certain set of shared values. We default to openness with our team, our customers, and everyone if possible. We love initiative — without arrogance or dictatorship. We work to create a place people enjoy showing up to work. That doesn’t mean ping pong tables and foosball (though we do try to have perks & fun), but it means people are friendly, non-political, working to build a good service but also a good place to work.

Do the work: Ideas and strategy are critical, but good execution makes them happen. We’re looking for someone who can help the team execute both from the perspective of being capable of guiding and organizing, but also someone who is hands-on themselves.

Additional Responsibilities needed for this role:

  • Recruit, coach, mentor, manage and lead a team of sales professionals to achieve yearly sales targets. This includes closing new business and expanding upon existing clientele.
  • Expand the customer success program to provide the best customer experience possible resulting in upsell opportunities and a high retention rate.
  • Develop effective sales strategies and deliver compelling product demonstrations and sales pitches.
  • Acquire and develop the appropriate sales tools to make the team efficient in their daily work flow.
  • Apply a thorough understanding of the marketplace, industry trends, funding developments, and products to all management activities and strategic sales decisions.
  • Ensure that sales department operations function smoothly, with the goal of facilitating sales and/or closings; operational responsibilities include accurate pipeline reporting and sales forecasts.
  • This position will report directly to the VP of Sales and will be staffed in our headquarters in San Mateo, CA.

Requirements:

  • 7 – 10+ years of successful sales leadership experience as measured by sales performance against goals.
    Experience in developing skill sets and providing career growth and opportunities through advancement of team members.
  • Background in selling SaaS technologies with a strong track record of success.
  • Strong presentation and communication skills.
  • Must be able to travel occasionally nationwide.
  • BA/BS degree required

Think you want to join us on this adventure?
Send an email to jobscontact@backblaze.com with the subject “Director of Sales.” (Recruiters and agencies, please don’t email us.) Include a resume and answer these two questions:

  1. How would you approach evaluating the current sales team and what is your process for developing a growth strategy to scale the team?
  2. What are the goals you would set for yourself in the 3 month and 1-year timeframes?

Thank you for taking the time to read this and I hope that this sounds like the opportunity for which you’ve been waiting.

Backblaze is an Equal Opportunity Employer.

The post Hiring a Director of Sales appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

[$] Unprivileged filesystem mounts, 2018 edition

Post Syndicated from corbet original https://lwn.net/Articles/755593/rss

The advent of user namespaces and container technology has made it possible
to extend more root-like powers to unprivileged users in a (we hope) safe
way. One remaining sticking point is the mounting of filesystems, which
has long been fraught with security problems. Work has been proceeding to
allow such mounts for years, and it has gotten a little closer with the
posting of a patch series intended for the 4.18 kernel. But, as an
unrelated discussion has made clear, truly safe unprivileged filesystem
mounting is still a rather distant prospect — at least, if one wants to do
it in the kernel.