Tag Archives: tech

Cheezball Rising: Maps and sprites

Post Syndicated from Eevee original https://eev.ee/blog/2018/07/15/cheezball-rising-maps-and-sprites/

This is a series about Star Anise Chronicles: Cheezball Rising, an expansive adventure game about my cat for the Game Boy Color. Follow along as I struggle to make something with this bleeding-edge console!

GitHub has intermittent prebuilt ROMs, or you can get them a week early on Patreon if you pledge $4. More details in the README!


In this issue, I get a little asset pipeline working and finally have a real map.

Previously: spring cleaning.
Next: resounding failure.

Recap

The last post only covered some minor problems (including, I grant you, being totally broken), so the current state of the game is basically unchanged from before.

A space cat roams around on a grassy background

That grass pattern, the grass sprite itself, and the color scheme are all hardcoded — written directly into the source code, by hand. If this game is going to get very far at all, I urgently need a better way to inject some art.

Constraints

The Game Boy imposes some fairly harsh constraints on the artwork — which is part of the charm! But now I have to figure out how to work within those constraints most effectively. Here’s what I’ve got to work with.

Bear in mind that I intend for the game to be based around 16×16, um, tiles. Okay, it’s extremely confusing that “tile” might refer either to the base size of the artwork or to the Game Boy’s native 8×8 tiles, so I’m going to call the art tiles and the Game Boy’s basic unit a character (which is what the manual does).

  • The background layer is a grid of 8×8 characters, each of which uses one of eight 4-color background palettes.

  • The object layer is a set of 8×16 character pairs, each of which uses one of eight 3-color object palettes. These palettes are 3-color because color 0 is always transparent.

  • No more than 40 objects can appear on screen at the same time. (There is a way to weasel past this limit, but it requires considerable trickery.)

  • No more than 10 objects can appear in the same row of pixels. (I believe this is a hard limit.)

  • There are three blocks of 256 chars each. I can divide this between the background and objects more or less however I want, though neither can have more than two blocks (= 512 chars).

I’m intending for the game to be based around a 16×16 grid, a fairly common size for the Game Boy. That makes me a little concerned about the per-row object limit — each entity will need to have two Game Boy objects side by side, so I’m really limited to only five entities sharing the same row of pixels. I can’t do much about that quite yet (and only have one entity anyway), but it’s likely to affect how I design maps and draw sprites.

The next biggest problem is colors. Each object palette can only have three colors, which in practice means a shadow/outline color, a highlight color, and a base color. This is why every NPC and overworld critter in Pokémon GSC and the Zeldas is basically monochromatic. They pull it off really well by making very effective use of the highlight and shadow colors.

Since 16×16 sprites are composed of multiple Game Boy objects, it’s possible to overcome this limit by giving each part of the sprite a different palette. Unfortunately, objects being 8×16 means the sprites are split vertically, when it would be most useful to have different colors for e.g. the head and body. I wish the Game Boy supported 16×8 objects! That’d help a ton with the per-row limit, too. Alas, a few decades too late to change it now.


As for the number of chars… well, let’s see. The whole screen is only 160×144, which is 20×18 or 360 chars, so I could allocate two blocks to the background and have 512 — more than enough to cover the entire screen in unique chars! (I expect one block to be more than enough for objects, since I can only show 80 object chars at once anyway.)

On the other hand, I’ll need to reserve some of that space for text and UI and whatnot, and each 16×16 tile is composed of four chars. If I very generously allocate a whole block to window dressing (enough for all of ISO-8859-1?), that leaves 256 chars, which is 64 tiles, which is a tileset that fits in an eight-by-eight square.

For comparison’s sake, even fox flux’s relatively limited tileset is a sixteen-tile square — four times as big. This feels a little dire.

But how can it be dire, when I have enough sprite space to fill the screen and then some?

Let’s see here. A pretty good chunk of the fox flux tileset is unused or outright blank. Some of these tiles are art for moving objects that happened to fit in the grid, and those wouldn’t be in the background tileset. And while all of the tiles are distinct, a lot of the basic terrain has some significant overlap:

A set of dirt tiles from fox flux, colored to indicate where different tiles have identical corners

All of the regions of the same color are identical. These 9 distinct tiles could fit into 20 chars if they shared the common parts, rather than the 36 required to naïvely cutting each one into four dedicated chars.

(The fox flux grid is 32×32, so everything is twice as big as it will be on the Game Boy, but you get the idea.)

I’m feeling a little better about this, especially knowing I do have enough space to cover the whole screen. Worst case, I could draw the map as though it were a single bitmap. I don’t want to have to rely on that if I can get away with it, though — I suspect I’d need to constantly load chars on the fly, and copying stuff around eats into my CPU budget surprisingly quickly.

Research

That does get me wondering: what, exactly, do the Oracle games do? I haven’t done any precise measurements, but I’m pretty sure they have more than sixty-four distinct map tiles throughout their large connected worlds. Let’s have a look!

Oracle of Ages and its live tilemap, in the graveyard, showing the graveyard tileset

Here I am in the graveyard near the start of Oracle of Ages. The “creepy tree” here is distinct and doesn’t really appear anywhere else, so I found it in the tile viewer (lower right) and will be keeping an eye on it. Note that only the left half of the face is visible; the right half is using the same tiles, flipped horizontally. (The colors are different because the tile viewer shows the literal colors, whereas the game itself is being drawn with a shader.)

Let’s walk left one screen.

Oracle of Ages and its live tilemap, outside of the graveyard

Now, this is interesting. The creepy tree is still on the screen here, so its tiles are naturally still loaded. But a bunch of tiles on the left — parts of the dungeon entrance and other graveyard things — have been replaced by town tiles. I’m several screens away from the town!

The next screen up has no creepy trees, but its tiles remain. Of course, they’d have to, since the creepy tree is still visible during a transition. I have to go left from there before the tree disappears:

Oracle of Ages and its live tilemap, with tiles spelling SHOP clearly visible

Wow! At a glance, this looks like enough tiles to draw the entire town.

This is fascinating. The Oracle games have several transitions between major areas, marked by fade-outs or palette changes — the purple-tinted graveyard is an obvious example. But it looks like there are also minor transitions that update the tileset while I’m still several screens away from where those tiles are used. The screens around the transition only use common tiles like grass and regular trees, so I never notice anything is happening.

That’s cute, clever, and an easy way to make screen transitions work without having to figure out what tiles are becoming unused as they slide off the screen!

At this point I realize I may be getting ahead of myself. Screen transitions? I don’t have a map yet! Hell, I don’t even have a camera. Time to back up and make something I can build on.

Designing a tileset

I’m pretty tired of manually translating art into bits. It’s 2018, dammit. I want to use all the regular tools I would use for this, I want the Game Boy’s limitations to be expressed as simply as possible, and I want minimal friction between the source artwork and the game.

Here’s my idea. I know I only have 8 palettes to work with, so I’m decreeing that tilesets will be stored as paletted PNGs. The first four colors in the image palette will become the first Game Boy palette; the next four colors become the second Game Boy palette; and so on. If I then resize Aseprite’s palette panel to be four colors wide, I’ll have an instant view of all my available combinations of colors.

This already has some problems — for starters, if the same color appears in multiple palettes (which will almost certainly happen, for the sake of cohesion), I’m very likely to confuse the hell out of myself. I also have no idea how to extend this into multiple tilesets, but for now I’ll pretend the entire game world only uses a single tileset.

I could instead dynamically infer the palettes based on what combinations of colors are actually used, but after more than a couple tiles, it would be a nightmare for a human to keep track of what those combinations are. With this approach, all a human needs to do is color-drop a pixel from a particular tile and look at what row the color’s in.

After a quick jaunt into the pixel mines, here are some tiles.

A small set of pastel yellow moon tiles

Or, as viewed in Aseprite:

The same set of tiles, as seen in an editor, with the four-color palette visible

That’s only one palette, but hopefully you can see what I’m going for here. It’s enough to get started.

At this point, I started writing a little Python script that used Pillow to inspect the colors and pixels and dump them out to rgbasm-flavored source code. The script itself is not especially interesting: run through each 8×8 block of pixels, look at each pixel’s palette index, mod 4 to get the index within the Game Boy palette, print out as backtick literals. (I could spit out raw binary data, but I wanted to be able to inspect the intermediate form easily. Maybe later.)

The results:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
SECTION "Map dumping test", ROM0
TEST_PALETTES:
    dw %0101011110111101
    dw %0101011100011110
    dw %0100101010111100
    dw %0100011001111000
    ; ... enough zeroes to make eight palettes ...
; sorry, in the script I was calling them "tiles", not "chars"
TEST_TILES:
    ; tile 0 at 0, 0
    dw `00001000
    dw `00000000
    dw `00100000
    dw `00000000
    dw `00000000
    dw `00000000
    dw `20000000
    dw `20000002
    ; ... etc ...

And hey, I already have code that can load palettes and chars, so all I have to do is swap out the old labels for these ones.

Now I have a tileset I can load into the game, which is very exciting, except that I can’t see any of them because I still don’t have a map. I could draw a test map by hand, I suppose, but the whole point of this exercise was to avoid ever doing that again.

Drawing a map

In keeping with the “it’s 2018 dammit” approach, I elect to use Tiled for drawing the maps. I’ve used it for several LÖVE games, and while its general-purposeness makes it a little clumsy at times, it’s flexible enough to express basically anything.

I make a tileset and create a map. I choose 256×256 pixels (16×16 tiles), the same size as the Game Boy screen buffer, and fill it with arbitrary terrain. In retrospect, I probably should’ve made it the size of the screen, since I still don’t have a camera. Oh, well.

Here, I hit a minor roadblock. I want to do as much work as possible upfront, so I want to store the map in the ROM as chars, not tiles. That means I need to know what chars make up each tile, which is determined by the script that converts the image to char data. Multiple maps might use the same tileset, and a map might use multiple tilesets, so it seems like I’ll need some intermediate build assets with this information…

(In retrospect again, I realize that the game may need to know about tiles rather than just chars, since there’ll surely be at least a few map tiles that act like entities — switches and the like — and those need to function as single units. I guess I’ll work that out later.)

This is all looking like an awful lot of messing around (and a lot of potential points of failure) before I can get anything on the dang screen. I waffle for a bit, then decide to start with a single step that simultaneously dumps the tiles and the map. I can split it up when I actually have more than one of either.

You can check out the resulting script if you like, but again, I don’t think it’s particularly interesting. It enforces a few more constraints than before, and adds a TEST_MAP_1 label containing all the char data, row by row. Loading that into VRAM is almost comically simple:

1
2
3
4
5
    ; Read from the test map
    ld hl, $9800
    ld de, TEST_MAP_1
    ld bc, 1024
    call copy16

The screen buffer is 32×32 chars, or 1024 bytes. As you may suspect, copy16 is like copy, but it takes a 16-bit count in bc.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
; copy bc bytes from de to hl
; NOTE: bc must not be zero
copy16:
    ld a, [de]
    inc de
    ld [hl+], a
    dec bc
    ; dec bc doesn't set flags, so gotta check by hand
    ld a, b
    or a, c
    jr nz, copy16
    ret

Hm. It’s a little harder to justify the bc = 0 case as a feature here, since that would try to overwrite every single byte in the entire address space. Don’t do that, then.

Anise, in-game, walking on the moon tiles

Now, at long long last, I have a background with some actual art! It’s starting to feel like something! I’ve even got something resembling a workflow.

My desktop, showing the moon tiles in an image editor, the map put together in Tiled, and the game running in mGBA

All in a day’s work. Good time to call it, right?

Except

I just wrote this char loading code…

And there’s still one thing still hardcoded…

I wonder if I could do something about that…?

Sprites

Above, I conspicuously did not mention how I integrated the Python script into the build system. And, well, I didn’t do that. I ran it manually and put it somewhere and committed it all as-is. You currently (still!) can’t actually build the game without repeating my steps. You can’t even just put the output in the right place, because you also have to delete some debug output from the middle of the file.

It gets worse! Here’s how.

I have some Anise walking sprites, too, drawn in Aseprite. They’re pretty cute and I’d love to have them in the game, now that I have some Real Art™ for the background.

Star Anise, walking forwards

Why not throw these at the same script and hack them into animating?

Unfortunately, this introduces a bit of manual work, as animation often does. (My kingdom for a way to embed a small simple animation in a larger spritesheet in Aseprite!) I’ve typically animated every critter in its own Aseprite file — or stacked several vertically in the same file when their animations are similar enough — and then exported as a sheet with the frames running off horizontally. You can see this at work in fox flux, e.g. on its critter sheet.

But Star Anise introduces a wrinkle that prevents even that slightly clumsy workflow from working.

You may have noticed that the walking sprite above blows the color budget considerably, using a whopping five colors. The secret is that Anise himself fits in a 16×16 square, and then his antenna is a third 8×16 sprite drawn on top. I can’t simply export him as a spritesheet, because the antenna needs to be separate, and it’s not even aligned to the grid. It doesn’t even stay in the same place consistently!

I could maybe hack something together that would automatically pull the incompatible pixels into a separate sprite. I might need to, since — spoiler alert — there are an awful lot of Lunekos in this game. For now, though, I did the dumbest thing that works and copied his frames to their own sheet by hand.

Star Anise's walking frames laid out in a spritesheet

The background is actually cyan, not transparent. I had to do this because my setup expects multiple sets of four colors — the first color in an object palette is still there, even if it’s ignored — and only one color in an indexed PNG can be transparent. (Don’t @ me about PNG pixel formats.) I could’ve adjusted it to work with sets of three colors and put the transparent one at the end so the palette column trick still worked, but… this was easier.

Here’s the best part: I took the main function from my tile loading script, copy-pasted it within the same file, and edited the copy to dump these sprites sans map. So now not only is there no build system, but half of the loading script is inaccessible! Sorry. We’re getting into experiment territory and I am going to start making a lot of messes while I figure out what I actually want.

Using these within the game was just as easy as before — replace some labels with new ones — and the only real change was to use a third OAM slot for the antenna. (The antenna has to appear first; when sprites overlap, the one with the lowest index appears on top.)

That did make updating OAM a little clumsy; you may recall that before, I loaded the x and y positions into b and c, updated them, then wrote them back into OAM:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    ; set b/c to the y/x coordinates
    ld hl, oam_buffer
    ld b, [hl]
    inc hl
    ld c, [hl]
    bit BUTTON_LEFT, a
    jr z, .skip_left
    dec c
.skip_left:
    bit BUTTON_RIGHT, a
    jr z, .skip_right
    inc c
.skip_right:
    bit BUTTON_UP, a
    jr z, .skip_up
    dec b
.skip_up:
    bit BUTTON_DOWN, a
    jr z, .skip_down
    inc b
.skip_down:
    ld [hl], c
    dec hl
    ld [hl], b
    ld a, c
    add a, 8
    ld hl, oam_buffer + 5
    ld [hl], a
    dec hl
    ld [hl], b

The above approach required that I hardcode the 8-pixel offset between the left and right halves. With the antenna in the mix, I would’ve had to hardcode another more convoluted offset, and I didn’t like the sound of that. So I changed it to inc and dec the OAM coordinates directly and immediately:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
    ; Anise update loop
    ; set b/c to the y/x coordinates
    ld bc, 4
    bit BUTTON_LEFT, a
    jr z, .skip_left
    ld hl, oam_buffer + 1
    dec [hl]
    add hl, bc
    dec [hl]
    add hl, bc
    dec [hl]
.skip_left:
    ; ... etc ...

Eventually I should stop doing this and have an actual canonical x/y position for Anise somewhere. But I didn’t do that yet.

I did also take this opportunity to change my LCDC flags so that object chars start counting from zero at $9000, fixing the misunderstanding I had before. That’s nice.

Anyway, tada, Star Anise can slide around, but now with his antenna.

Not good enough.

Animating

It’s time to animate something. And this time around, all I’ve got are bytes to work with. Oh, boy!

Right out of the gate, I have two options. I could load all of Anise’s sprites into VRAM upfront and change the char numbers in OAM to animate him, or I could reserve some specific chars and overwrite them to animate him.

The first choice makes sense for an entity that might exist multiple times at once, like enemies or… virtually anything in the game world, really. But there’s only ever one player, and he’s likely to have a whole lot of spritework, which I would prefer not to have clogging up my char space for the entire duration of the game. So while I might use the other approach for most other things, I’m going to animate Anise by overwriting the actual graphics. Every frame.

First things first. I’m going to need some state, which I’ve been avoiding by relying on OAM. At the very least, I need to know which way Anise is facing — which isn’t necessarily the direction he’s moving, because he should keep his facing when he stops. I also need to know which animation frame he’s on, and how many LCD frames are left until he should advance to the next one.

Let’s refer to the time between vblanks as a “tic” for now, to avoid the ambiguity of a “frame” when talking about animation.

A good start, then, would be some constants.

1
2
3
4
5
6
FACING_DOWN   EQU 0
FACING_UP     EQU 1
FACING_RIGHT  EQU 2
FACING_LEFT   EQU 3

ANIMATION_LENGTH EQU 5

ANIMATION_LENGTH is the length of every frame. I don’t especially want to give every frame its own distinct duration if I can avoid it; this will be complicated enough as it is. I fiddled with the frame duration in Aseprite for a bit and landed on 83ms as a nice speed, and that’s 5 tics.

I also need a place for this state, so I add some more stuff to my RAM block.

1
2
3
4
5
6
anise_facing:
    db
anise_frame:
    db
anise_frame_countdown:
    db

And initialize it in setup.

1
2
3
4
    ld a, FACING_DOWN
    ld [anise_facing], a
    ld a, ANIMATION_LENGTH
    ld [anise_frame_countdown], a

Presumably, one day, I’ll have multiple entities, and they’ll all share a similar structure, which I’ll have to traverse manually. For now, it’s easier to follow the code if I give every field its own label.

I have four levels of hierarchy here: the spriteset (which for now is always Anise’s), the pose (I only have one: walking), the facing, and the frame. I need to traverse all four, but luckily I can ignore the first two for now.

I don’t want to animate Anise when he’s not moving, so I changed the OAM updating code to also ld d, 1 if there’s any movement at all, and skip over all the animation stuff if d is still zero.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
    ; ... read input ...

    ; This was before I knew the 'or a' trick; these two ops
    ; could be replaced with 'xor a; or d'
    ld a, d
    cp a, 0
    jp z, .no_movement

    ; ... all the animation code will go here ...

.no_movement:
    ; and after this we repeat the main loop

This does have the side effect that Anise will simply freeze in mid-walk when stopped, rather than returning to his standing pose. I still haven’t fixed that; I could special-case it, but I usually treat “standing” as its own one-frame animation, so it feels like something that ought to come when I implement poses.

Next I decrement the countdown, which is the number of tics left until the frame ought to change. If this is nonzero, I don’t need to do anything.

1
2
3
4
5
6
    ld a, [anise_frame_countdown]
    dec a
    ld [anise_frame_countdown], a
    jp nz, .no_movement
    ld a, ANIMATION_LENGTH
    ld [anise_frame_countdown], a

Again, this isn’t actually right. If Anise’s state changes, such as between standing and walking, then this should be ignored because he’s switching to a new animation. But this is a pose thing again, so I’m deferring it until later.

Next I need to advance the current frame. I don’t have modulo on hand and even simple ifs are kind of annoying, so I was naughty here and used bitops to roll from frame 3 to frame 0. This would obviously not work if the number of frames were not a power of two.

1
2
3
4
    ld a, [anise_frame]
    inc a
    and a, 4 - 1
    ld [anise_frame], a

Yet again, if Anise changes direction, the frame should be reset to zero… but it ain’t.

Now, let’s think for a second. I know what frame I want. I have a label for the upper-left corner of the spritesheet, and I want to get to the upper-left corner of the appropriate frame. Each frame has 3 objects; each object has 2 chars; each char is 16 bytes.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
    ld hl, ANISE_TEST_TILES
    ; Skip ahead 3 sprites * the current frame
    ld bc, 3 * 2 * 16
    ; Remember, zero iterations is also possible
    or a
    jr z, .skip_advancing_frame
.advance_frame:    
    add hl, bc
    dec a
    jr nz, .advance_frame
.skip_advancing_frame:
    ; Copy the sprites into VRAM
    ; They're consecutive in both the data and VRAM, so only
    ; one copy is necessary.  And bc is already right!
    ld d, h
    ld e, l
    ld hl, $8000
    call copy16

Hey, look at that!

Star Anise walking around in-game, now animated

Only one small problem: I forgot about facing, so Anise will always face forwards no matter how he moves. Whoops!

Facing

I need to actually track which way Anise is facing, which is a surprisingly subtle question. He might even be facing away from his own direction of movement, if for example he was thrown backwards by some external force.

A decent first approximation is to use the last button that was pressed. (That’s still not quite right — if you hold down, hold down+right, and then release right, he should obviously face down. But it’s a start.)

I don’t yet track which buttons were pressed this frame, but it’s easy enough to add. While I’m at it, I might as well track which buttons were released, too. I amend the input reading code thusly, based on the straightforward insight that a button was pressed this frame iff it is currently 1 and was previously 0.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
    ; a now contains the current buttons
    ld hl, buttons
    ld b, [hl]                  ; b <- previous buttons
    ld [hl], a                  ; a -> current buttons
    cpl
    and a, b
    ld [buttons_released], a    ; a = ~new & old, i.e. released
    ld a, [hl]                  ; a <- current buttons
    cpl
    or a, b
    cpl
    ld [buttons_pressed], a     ; a = ~(~new | old), i.e. pressed

I like that cute trick for getting the pressed buttons. I need a & ~b, but cpl only works on a, so I would’ve had to juggle a bunch of registers. But applying De Morgan’s law produces ~(~a | b), which only requires complementing a. (Full disclosure: I didn’t actually try register juggling, and for all I know it could end up shorter somehow.)

Next I check the just-pressed buttons and updating facing accordingly. It looks a lot like the code for checking the currently-held buttons, except that I only use the first button I find.

1
2
3
4
5
6
7
8
    ld hl, anise_facing
    ld a, [buttons_pressed]
    bit BUTTON_LEFT, a
    jr z, .skip_left2
    ld [hl], FACING_LEFT
    jr .skip_down2
.skip_left2:
    ; ... you get the idea ...

And finally, amend the sprite choosing code to pick the right facing, too.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
    ld hl, ANISE_TEST_TILES

    ; Skip ahead a number of /rows/, corresponding to facing
    ld a, [anise_facing]
    and a, %11                      ; cap to 4, just in case
    jr z, .skip_stride_row
    ; This is like before, but times 4 frames
    ld bc, 4 * 3 * 2 * 16
.stride_row:
    add hl, bc
    dec a
    jr nz, .stride_row
.skip_stride_row:

    ; Bumping the frame here is convenient, since it leaves the
    ; frame in a for the next part
    ld a, [anise_frame]
    inc a
    and a, 4 - 1
    ld [anise_frame], a

    ; ... continue on with picking the frame ...

Hardcoding the number of frames here is… unfortunate. I should probably flip the spritesheet so the frames go down and each column is a facing; then there’ll always be a fixed number of columns to skip over.

But who cares about that? Look at Anise go! Yeah!

Star Anise walking around in-game, now animated in all four directions

Well, yes, there is one final problem, which is that the antenna is misaligned when walking left or right… because its positioning is different than when walking up or down, and I don’t have any easy way to encode that at the moment. It’s still like that, in fact. I’m sure I’ll fix it eventually.

More vblank woes

I didn’t run into this problem until a little while later, but I might as well mention it now. The above code writes into VRAM in the middle of updating entities — updating them very simply, perhaps, but updating nonetheless. If that updating takes longer than vblank, the write will fail.

I expected this, though not quite so soon. It’s a disadvantage of swapping the char data rather than the char references: 32× more writing to do, which will take 32× longer. The solution is similar to what I do for OAM: defer the write until the next vblank. I’m already doing that with Anise’s position, anyway, and it makes no sense to have his position and animation updated on different frames.

I ended up special-casing this for Anise, though it wouldn’t be too hard to extend this into a queue of tiles to copy. It’s nothing too world-shaking; I just store the address of Anise’s current sprite in RAM, then copy it over during vblank, just after the OAM DMA.

I did try doing this with one of the Game Boy Color’s new features, general-purpose DMA, which can copy from basically anywhere in ROM or RAM to basically anywhere in VRAM. It involves five registers: you write the source address in the first two, the destination in the next two, and the length in the fifth, which triggers the copy. The CPU simply freezes until the copy is done, so there are no goofy timing issues here.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
    ld hl, anise_sprites_address
    ld a, [hl+]
    ld [rHDMA1], a
    ld a, [hl]
    ld [rHDMA2], a
    ld a, HIGH($0000)
    ld [rHDMA3], a
    ld a, LOW($0000)
    ld [rHDMA4], a
    ; To copy X bytes, write X / 16 - 1 to this register
    ld a, (32 * 3) / 16 - 1
    ld [rHDMA5], a

General-purpose DMA can copy 16 bytes every 8 cycles, or ½ cycle per byte. The fastest possible manual copy would be an unrolled series of ld a, [hl+]; ld [bc], a; inc bc which takes a whopping 6 cycles per byte — twelve times slower! This is a neat feature.

FYI, it’s also possible to have a copy done piecemeal during hblanks, though that sounds a bit fragile to me.

Future work

I’ve laid some very basic groundwork here, and there’s plenty more to do, which I will get back to later! It’s just me hacking all this together, after all, and I like flitting between different systems.

I will definitely need to figure out how the heck multiple tilesets work and when they get switched out. How do I even use multiple tilesets, each with its own set of palettes? What’s the workflow if I want to use the same tiles with several different palettes, like how the graveyard in Oracle of Ages is tinted purple? And I didn’t even implement character de-duplication yet… which will require some metadata for each tile… aw, geez.

And I still haven’t fixed the build system! Maybe you can understand why I’m hesitant to impose more structure on this idea quite yet.

To be continued

That brings us to commit 59ff18. Except for a commit about the build that I skipped. Whatever. This post has been a little more draining to write, perhaps because it forced me to confront and explain a bunch of hokey decisions.

Next time: resounding failure!

Cheezball Rising: Spring cleaning

Post Syndicated from Eevee original https://eev.ee/blog/2018/07/13/cheezball-rising-spring-cleaning/

This is a series about Star Anise Chronicles: Cheezball Rising, an expansive adventure game about my cat for the Game Boy Color. Follow along as I struggle to make something with this bleeding-edge console!

GitHub has intermittent prebuilt ROMs, or you can get them a week early on Patreon if you pledge $4. More details in the README!


In this issue, I tidy up some of the gigantic mess I’ve made thusfar.

Previously: writing a main loop, and finally getting something game-like.
Next: sprite and map loading.

Recap

After only a few long, winding posts’ worth of effort, I finally have a game, if you define “game” loosely as a thing that reacts when you press buttons.

A space cat roams around on a grassy background

Beautiful. But to make an omelette, you need to break a few eggs, and if it’s your first omelette then you might break some glassware too. As tiny as this game is, a couple things could use improvement.

Also, for narrative purposes, it’s much more interesting to put all these miscellaneous fixes together, rather than interrupting other posts with them. I didn’t actually do all this work in one lump in this order. Apologies to the die-hard non-fiction crowd.

It’s totally broken

Ah, the elephant in the room. The end of the previous post aligned with the first demo build, but if you downloaded it and tried to play it, you may have seen something that looks more like this:

Similar to the previous image, but with obvious graphical corruption

I said in the beginning that I liked mGBA and would be developing against it. That’s still true — it’s open source (and I’ve actually read some of it), it’s cross-platform, and it has some debug tools built in.

I also said that emulators are primarily designed to accept correct games, not necessarily to reject incorrect games. And that’s still very true.

I discovered this problem myself a little later (after the events of the next post), while shopping around a bit for emulators explicitly focused on accuracy. The one I keep being told to use is bgb, but it’s for Windows and Wine is kind of annoying, so I was exploring my other options; I found SameBoy (primarily for Mac, but with Linux and Windows builds sans debug features) and Gambatte (cross-platform, and the core for RetroArch’s Game Boy emulation). All three of them looked like the screenshot above.

Something was going very wrong when writing to VRAM. You can’t write to VRAM while the LCD is redrawing, so the most obvious cause is that… well… maybe the LCD is redrawing during my setup code.

Remember, on an actual Game Boy, the system doesn’t immediately start running what’s on the cartridge — it scrolls in the Nintendo logo first (or on a Color, does a fancier logo with a cool fanfare). That’s done by a tiny internal program called the boot ROM, and the state of the LCD when the boot ROM hands over control is undefined. I’m sure it’s consistent, but it’s not anything in particular, and for all I know it might be when the LCD is halfway through a redraw.

(Side note: I am violating Nintendo’s game submission requirements by consistently referring to it as a “cartridge” when in fact it is properly called a Game Pak. My bad.)

So what we’re seeing above is the result of VRAM becoming locked and unlocked as the LCD draws (remember, after every row is an hblank, during which time VRAM is accessible), while I’m trying to copy blocks of data there. In fact, every emulator I’ve tried shows a slightly different form of corruption, since this problem is very sensitive to timing accuracy. Super interesting!

I could wait for vblank and try to squeeze in all my setup code there, maybe even split across several vblanks. But since this is setup code and doesn’t run during gameplay, there’s a much easier solution: turn the screen off. That’s done with a bit in the LCDC register, which I currently configure at the end of my setup code; all I need to do is move that to the beginning and clear the appropriate bit instead.

1
2
    ld a, %00010111  ; $91 plus bit 2, minus bit 7
    ld [$ff40], a

Then, of course, set it again once I’m done. I did this with a couple macros, since it’s only a few instructions and it seems like the kind of thing I might need again later.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
DisableLCD: MACRO
    ld a, [$ff40]
    and a, %0111111
    ld [$ff40], a
ENDM

EnableLCD: MACRO
    ld a, [$ff40]
    or a, %10000000
    ld [$ff40], a
ENDM

; and, of course, stick an EnableLCD at the end of setup code

Note that when the screen is off, it’s off, and there are no vblank interrupts or anything else that might be triggered by the screen’s behavior. So, you know, don’t wait for vblank while the screen’s off. When the screen turns back on, it immediately starts redrawing from the first row, so don’t try to use VRAM right away either. Finally, on the original Game Boy, do not turn off the screen when it’s not in vblank, or you might physically damage the screen. It’s fine on the Game Boy Color, but… hell, I’m gonna edit this to wait for vblank anyway. Feels kinda inappropriate to abruptly turn off the screen halfway through drawing.

Anyway, that solves my goofy corruption problems, and now the game looks the same on all of these emulators! I also reported this misbehavior, and it’s since been fixed, so recent dev builds of mGBA also correctly render garbage for the first release. See, by not targeting the most accurate emulators, I’ve caused another emulator to become more accurate!

hardware.inc

I mentioned last time that I’d adopted hardware.inc. That’s in large part because I keep producing monstrosities like the previous snippet. Here are those macros with some symbolic constants:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
DisableLCD: MACRO
    ld a, [rLCDC]
    and a, $ff & ~LCDCF_ON
    ld [rLCDC], a
ENDM

EnableLCD: MACRO
    ld a, [rLCDC]
    or a, LCDCF_ON
    ld [rLCDC], a
ENDM

A breath of fresh air!

The $ff & is necessary because the argument needs to fit in a byte, but rgbasm’s integral preprocessor type is wider than a byte. I suppose I could also use LOW() here, or maybe there’s some other more straightforward solution.

Rearranging the buttons

In the previous post, I read the button states and crammed them into a single byte. I had a choice of whether to put the dpad low or the buttons low, but it didn’t seem to matter, so I picked arbitrarily: buttons high, dpad low.

It turns out I chose wrong! Also, it turns out there’s a “wrong” here! I’ve heard two compelling reasons to do it the other way. For one, hardware.inc contains constants for the bit offsets of the buttons, and it assumes the dpad is high. Why is this arbitrary data layout decision embedded in a list of hardware constants? Possibly for the second reason: on the GBA, input is available as a single word, and the lowest byte contains bits for all the buttons on the Game Boy — in the same order, with the dpad high.

So I’m switching this around and using hardware.incs constants. Easy change.

Fixing vblank

My original approach to waiting for vblank seemed simple enough: loop until vblank_flag is set, clear it, then continue on.

I’ve made a slight oversight here: what if the main loop does take longer than a frame? Then a vblank interrupt will fire in the middle of it and harmlessly set vblank_flag. But when the loop finally finishes and goes to wait for vblank again, the flag will already be set, and it’ll continue on immediately — regardless of the state of the screen! Whoops.

Again, the fix is simple: clear the flag before beginning to wait.

And while I’m at it, I see other uses for waiting for vblank in the near future, so I may as well pull this out into a function.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
; idle until next vblank
wait_for_vblank:
    xor a                       ; clear the vblank flag
    ld [vblank_flag], a
.vblank_loop:
    halt                        ; wait for interrupt
    ld a, [vblank_flag]         ; was it a vblank interrupt?
    and a
    jr z, .vblank_loop          ; if not, keep waiting
    ret

Copy function

So far, I’ve done an awful lot of runtime copying by using the preprocessor. Consider the code for copying the DMA routine into HRAM:

1
2
3
4
5
6
7
8
    ; Copy the little DMA routine into high RAM
    ld bc, dma_copy
    ld hl, $ff80
    REPT dma_copy_end - dma_copy
    ld a, [bc]
    inc bc
    ld [hl+], a
    ENDR

This will repeat the ld/inc/ld dance 13 times in the built ROM. Which is fine, except that I’m about to have places where I do much more copying, and there’s only so much space in the ROM, and this is kind of ridiculous. So I guess I will finally write a copy function.

I’m calling it copy, not memcpy. What else am I going to copy, if not memory?

Attempt number 1 looked like this:

1
2
3
4
5
6
7
8
; copy d bytes from bc to hl
copy:
    ld a, [bc]
    inc bc
    ld [hl+], a
    dec d
    jr z, copy
    ret

I was then informed that it’s more idiomatic to use de as the source address and c as the count, possibly for some reason relating to the NES or SNES? I don’t remember. I’m totally on board for using c to mean a count, though, and started doing that elsewhere.

I went to change that, and actually make use of this function, and lo! I discovered a colossal bug. That last line, jr z, copy, will loop only if d was just decremented to zero. So this function will only ever copy one byte, unless you asked to copy only one byte, in which case it copies two.

This is not the first time I’ve gotten a condition backwards. I’ll get used to it eventually, I’m sure.

Oh, one other minor problem: if you ask to copy zero bytes, you’ll actually copy 256, since the zero check only comes after the decrement. (This is a recurring annoyance, actually, and makes while loops surprisingly clumsy to express.) So far I’ve only ever needed to copy a constant amount, so this hasn’t been a problem, but… I’ll just leave a comment pretending it’s a feature.

1
2
3
4
5
6
7
8
9
; copy c bytes from de to hl
; NOTE: c = 0 means to copy 256 bytes!
copy:
    ld a, [de]
    inc de
    ld [hl+], a
    dec c
    jr nz, copy
    ret

And here it is in action:

1
2
3
4
5
    ; Copy the little DMA routine into high RAM
    ld de, dma_copy
    ld hl, $FF80
    ld c, dma_copy_end - dma_copy
    call copy

Cool.

Of course, this is now significantly slower than the original unrolled version. The original took 13 × (2 + 2 + 2) = 78 cycles; the function adds 6 cycles for the call, 4 cycles for the ret, and 13 × (1 + 3) = 52 for the counting and jumping. As c goes to infinity, the function takes about ⅔ longer than unrolling.

If I feel like it, I could mitigate this somewhat by partially unrolling. First I’d mask off some lower bits of c — say, the lowest two — and copy that many bytes. Now the amount of copying left is a multiple of four, so I could shift c right twice and have another loop that copies four bytes at a time, amortizing the cost of the decrement and jump.

It’s not urgent enough for me to want to bother yet, and it’ll make relatively little difference for small copies like this DMA one, but I’m strongly considering it for copying a 16-bit amount.

Reset vectors

Now I have a couple utility functions like copy and wait_for_vblank. I don’t really care where they go, so I put them in their own SECTION and let the linker figure it out.

It took a while for me to notice where, exactly, the linker had put them: at $0000! These functions are small, and I have nothing explicitly placed before the interrupt handlers (which begin at $0040), so rgblink saw some empty space and filled it.

The thing is, the Game Boy has eight instructions of the form rst $xx that act as fast calls — each one jumps to a fixed low address (a “reset vector”), using less time and space than a call would. And those fixed $xx addresses are… $00, and every eight bytes afterwards.

I don’t have any immediate use for these — eight bytes isn’t a lot, though I guess copy could fit in there — but I probably don’t want arbitrary code ending up where they go, so for now I’ll stub them out like I stubbed out the interrupt handlers.

(I have been advised of one very good use for reset vectors: putting a crash handler at $38. Why? Because rst $38 is encoded as $ff, which is a fairly common byte to encounter if you accidentally jump into garbage. A lot of the Game Boy’s RAM is even initialized to $ff at startup.)

Idioms

I’m still discovering what’s considered idiomatic, but here are a couple tidbits.

The set of instructions is a little scattershot as far as arguments go. Several times early on, I wrote stuff like this:

1
2
3
  ld hl, some_address
  ld a, 133
  ld [hl], a

But I overlooked that there are instructions for both ld [hl], n8 and ld [n16], a, so the above can be reduced to two lines. There’s no such thing as ld [n16], n8, though.

A surprising number of instructions can use [hl] directly as an operand — even inc and dec, combining fetch/mutate/store into a single instruction.

xor a is twice as short and twice as fast as ld a, 0. I mean, we’re talking about a single byte and single cycle here, but no reason not to.

(xor a really means xor a, a, but since every boolean op instruction takes a as the first argument anyway, it can be omitted. I don’t like to omit it in most cases, since xor b doesn’t mention a at all and that seems misleading, but it feels appropriate when combining a with itself.)

or a (equivalently, and a) is a quick way to test whether a is zero, since boolean ops set the zero flag.

Color

This is neither here nor there, but since this post began with emulator differences, here’s another one.

The screen you’re reading this on is almost certainly backlit, but the original Game Boy Color screen was not. A fully white pixel on a Game Boy Color is turned off — it’s the color of the screen itself, in which you can probably see your own reflection.

Which raises a tricky question: what color is that? The game thinks it’s pure white, but the screen was a sort of pale yellow. So how should it be rendered in an emulator, on a modern backlit LCD monitor?

Compounding this problem is that Game Boy Color games can also run on the Game Boy Advance, which showed the colors yet slightly differently. And, of course, even monitors may be calibrated differently, in which case it all goes out the window.

It’s interesting to see different emulators’ opinions of how to render color:

The same screenshot, seen in several different emulators with different color schemes

This is exactly the same ROM. The top left is mGBA out of the box, which shows colors completely unaltered — usually fairly saturated. The top right is mGBA with its “gba-colors” shader enabled, which is supposed to replicate how colors appear on a GBA screen, but seems passingly similar to a GBC too. Then on the bottom are two emulators renowned for their accuracy, here wildly disagreeing with each other.

My Game Boy Color is currently in a box somewhere, and until I can find it, I can’t be sure who’s closer. All of these are perfectly fine interpretations of the same art, though.

I may or may not use the “gba-colors” shader, and may or may not fiddle with mGBA’s color settings over time. If the colors vary a bit in future screenshots, that’s probably why.

To be continued

This post doesn’t really correspond to a particular commit very well, since it’s all little stuff I did here and there. I hope you’ve enjoyed the breather, because it’s all downhill from here. In a good way, I mean. Like a rollercoaster.

Next time: map and sprite loading, which will explain how I got from grass to the moon texture in the screenshots above!

Cheezball Rising: Main loop, input, and a game

Post Syndicated from Eevee original https://eev.ee/blog/2018/07/05/cheezball-rising-main-loop-input-and-a-game/

This is a series about Star Anise Chronicles: Cheezball Rising, an expansive adventure game about my cat for the Game Boy Color. Follow along as I struggle to make something with this bleeding-edge console!

GitHub has intermittent prebuilt ROMs, or you can get them a week early on Patreon if you pledge $4. More details in the README!


In this issue, I fill in the remaining bits necessary to have something that looks like a game.

Previously: drawing a sprite.

Recap

So far, I have this.

A very gaudy striped background with half a cat on top

It took unfathomable amounts of effort, but it’s something! Now to improve this from a static image to something a bit more game-like.

Quick note: I’ve been advised to use the de facto standard hardware.inc file, which gives symbolic names to all the registers and some of the flags they use. I hadn’t introduced it yet while doing the work described in this post, but for the sake of readability, I’m going to pretend I did and use that file’s constants in the code snippets here.

Interrupts

To get much further, I need to deal with interrupts. And to explain interrupts, I need to briefly explain calls.

Assembly doesn’t really have functions, only addresses and jumps. That said, the Game Boy does have call and ret instructions. A call will push the PC register (program counter, the address of the current instruction) onto the stack and perform a jump; a ret will pop into the PC register, effectively jumping back to the source of the call.

There are no arguments, return values, or scoping; input and output must be mediated by each function, usually via registers. Of course, since registers are global, a “function” might trample over their values in the course of whatever work it does. A function can manually push and pop 16-bit register pairs to preserve their values, or leave it up to the caller for speed/space reasons. All the conventions are free for me to invent or ignore. A “function” can even jump directly to another function and piggyback on the second function’s ret, kind of like Perl’s goto &sub… which I realize is probably less common knowledge than how call/return work in assembly.

Interrupts, then, are calls that can happen at any time. When one of a handful of conditions occurs, the CPU can immediately (or, rather, just before the next instruction) call an interrupt handler, regardless of what it was already doing. When the handler returns, execution resumes in the interrupted code.

Of course, since they might be called anywhere, interrupt handlers need to be very careful about preserving the CPU state. Pushing af is especially important (and this is the one place where af is used as a pair), because a is necessary for getting almost anything done, and f holds the flags which most instructions will invisibly trample.

Naturally, I completely forgot about this the first time around.

The Game Boy has five interrupts, each with a handler at a fixed address very low in ROM. Each handler only has room for eight bytes’ worth of instructions, which is enough to do a very tiny amount of work — or to just jump elsewhere.

A good start is to populate each one with only the reti instruction, which returns as usual and re-enables interrupts. The CPU disables interrupts when it calls an interrupt handler (so they thankfully can’t interrupt themselves), and returning with only ret will leave them disabled.

Naturally, I completely forgot about this the first time around.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
; Interrupt handlers
SECTION "Vblank interrupt", ROM0[$0040]
    ; Fires when the screen finishes drawing the last physical
    ; row of pixels
    reti

SECTION "LCD controller status interrupt", ROM0[$0048]
    ; Fires on a handful of selectable LCD conditions, e.g.
    ; after repainting a specific row on the screen
    reti

SECTION "Timer overflow interrupt", ROM0[$0050]
    ; Fires at a configurable fixed interval
    reti

SECTION "Serial transfer completion interrupt", ROM0[$0058]
    ; Fires when the serial cable is done?
    reti

SECTION "P10-P13 signal low edge interrupt", ROM0[$0060]
    ; Fires when a button is released?
    reti

These will do nothing. I mean, obviously, but they’ll do even less than nothing until I enable them. Interrupts are enabled by the dedicated ei instruction, which enables any interrupts whose corresponding bit is set in the IE register ($ffff).

So… which one do I want?

Game loop

To have a game, I need a game loop. The basic structure of pretty much any loop looks like:

  1. Load stuff.
  2. Check for input.
  3. Update the game state.
  4. Draw the game state.
  5. GOTO 2

(If you’ve never seen a real game loop written out before, LÖVE’s default loop is a good example, though even a huge system like Unity follows the same basic structure.)

The Game Boy seems to introduce a wrinkle here. I don’t actually draw anything myself; rather, the hardware does the drawing, and I tell it what to draw by using the palette registers, OAM, and VRAM.

But in fact, this isn’t too far off from how LÖVE (or Unity) works! All the drawing I do is applied to a buffer, not the screen; once the drawing is complete, the main loop calls present(), which waits until vblank and then draws the buffer to the screen. So what you see on the screen is delayed by up to a frame, and the loop really has an extra “wait for vsync” step at 3½. Or, with a little rearrangement:

  1. Load stuff.
  2. Wait for vblank.
  3. Draw the game state.
  4. Check for input.
  5. Update the game state.
  6. GOTO 2

This is approaching something I can implement! It works out especially well because it does all the drawing as early as possible during vblank. That’s good, because the LCD operation looks something like this:

1
2
3
4
5
6
7
LCD redrawing...
LCD redrawing...
LCD redrawing...
LCD redrawing...
VBLANK
LCD idle
LCD idle

While the LCD is refreshing, I can’t (easily) update anything it might read from. I only have free control over VRAM et al. during a short interval after vblank, so I need to do all my drawing work right then to ensure it happens before the LCD starts refreshing again. Then I’m free to update the world while the LCD is busy.

First, right at the entry point, I enable the vblank interrupt. It’s bit 0 of the IE register, but hardware.inc has me covered.

1
2
3
4
5
main:
    ; Enable interrupts
    ld a, IEF_VBLANK
    ldh [rIE], a
    ei

Next I need to make the handler actually do something. The obvious approach is for the handler to call one iteration of the game loop, but there are a couple problems with that. For one, interrupts are disabled when a handler is called, so I would never get any other interrupts. I could explicitly re-enable interrupts, but that raises a bigger question: what happens if the game lags, and updating the world takes longer than a frame? With this approach, the game loop would interrupt itself and then either return back into itself somewhere and cause untold chaos, or take too long again and eventually overflow the stack. Neither is appealing.

An alternative approach, which I found in gb-template but only truly appreciated after some thought, is for the vblank handler to set a flag and immediately return. The game loop can then wait until the flag is set before each iteration, just like LÖVE does. If an update takes longer than a frame, no problem: the loop will always wait until the next vblank, and the game will simply run more slowly.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
SECTION "Vblank interrupt", ROM0[$0040]
    push hl
    ld hl, vblank_flag
    ld [hl], 1
    pop hl
    reti

...

SECTION "Important twiddles", WRAM0[$C000]
; Reserve a byte in working RAM to use as the vblank flag
vblank_flag:
    db

The handler fits in eight bytes — the linker would yell at me if it didn’t, since another section starts at $0048! — and leaves all the registers in their previous states. As I mentioned before, I originally neglected to preserve registers, and some zany things started to happen as a and f were abruptly altered in the middle of other code. Whoops!

Now the main loop can look like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
main:
    ; ... bunch of setup code ...

vblank_loop:
    ; Main loop: halt, wait for a vblank, then do stuff

    ; The halt instruction stops all CPU activity until the
    ; next interrupt, which saves on battery, or at least on
    ; CPU cycles on an emulator's host system.
    halt
    ; The Game Boy has some obscure hardware bug where the
    ; instruction after a halt is occasionally skipped over,
    ; so every halt should be followed by a nop.  This is so
    ; ubiquitous that rgbasm automatically adds a nop after
    ; every halt, so I don't even really need this here!
    nop

    ; Check to see whether that was a vblank interrupt (since
    ; I might later use one of the other interrupts, all of
    ; which would also cancel the halt).
    ld a, [vblank_flag]
    ; This sets the zero flag iff a is zero
    and a
    jr z, vblank_loop
    ; This always sets a to zero, and is shorter (and thus
    ; faster) than ld a, 0
    xor a, a
    ld [vblank_flag], a

    ; Use DMA to update object attribute memory.
    ; Do this FIRST to ensure that it happens before the screen starts to update again.
    call $FF80

    ; ... update everything ...

    jp vblank_loop

It’s looking all the more convenient that I have my own copy of OAM — I can update it whenever I want during this loop! I might need similar facilities later on for editing VRAM or changing palettes.

Doing something and reading input

I have a loop, but since nothing’s happening, that’s not especially obvious. Input would take a little effort, so I’ll try something simpler first: making Anise move around.

I don’t actually track Anise’s position anywhere right now, except for in the OAM buffer. Good enough. In my main loop, I add:

1
2
3
4
    ld hl, oam_buffer + 1
    ld a, [hl]
    inc a
    ld [hl], a

The second byte in each OAM entry is the x-coordinate, and indeed, this causes Anise’s torso to glide rightwards across the screen at 60ish pixels per second. Eventually the x-coordinate overflows, but that’s fine; it wraps back to zero and moves the sprite back on-screen from the left.

The half-cat is now sliding across the screen

Excellent. I mean, sorry, this is extremely hard to look at, but bear with me a second.

This would be a bit more game-like if I could control it with the buttons, so let’s read from them.

There are eight buttons: up, down, left, right, A, B, start, select. There are also eight bits in a byte. You might suspect that I can simply read an I/O register to get the current state of all eight buttons at once.

Ha, ha! You naïve fool. Of course it’s more convoluted than that. That single byte thing is a pretty good idea, though, so what I’ll do is read the input at the start of the frame and coax it into a byte that I can consult more easily later.

Turns out I pretty much have to do that, because button access is slightly flaky. Even the official manual advises reading the buttons several times to get a reliable result. Yikes.

Here’s how to do it. The buttons are wired in two groups of four: the dpad and everything else. Reading them is thus also done in two groups of four. I need to use the P1 register, which I assume is short for “player 1” and is so named because the people who designed this hardware had also designed the two-player NES?

Bits 5 and 6 of P1 determine which set of four buttons I want to read, and then the lower nybble contains the state of those buttons. Note that each bit is set to 1 if the button is released; I think this is a quirk of how they’re wired, and what I’m doing is extremely direct hardware access. Exciting! (Also very confusing on my first try, where Anise’s movement was inverted.)

The code, which is very similar to an example in the official manual, thus looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
    ; Poll input
    ; The direct hardware access is nonsense and unreliable, so
    ; just read once per frame and stick all the button states
    ; in a byte

    ; Bit 6 means to read the dpad
    ld a, $20
    ldh [rP1], a
    ; But it's unreliable, so do it twice
    ld a, [rP1]
    ld a, [rP1]
    ; This is 'complement', and flips all the bits in a, so now
    ; set bits will mean a button is held down
    cpl
    ; Store the lower four bits in b
    and a, $0f
    ld b, a

    ; Bit 5 means to read the buttons
    ld a, $10
    ldh [rP1], a
    ; Apparently this is even more unreliable??  No, really, the
    ; manual does this: two reads, then six reads
    ld a, [rP1]
    ld a, [rP1]
    ld a, [rP1]
    ld a, [rP1]
    ld a, [rP1]
    ld a, [rP1]
    ; Again, complement and mask off the lower four bits
    cpl
    and a, $0f
    ; b already contains four bits, so I need to shift something
    ; left by four...  but the shift instructions only go one
    ; bit at a time, ugh!  Luckily there's swap, which swaps the
    ; high and low nybbles in any register
    swap a
    ; Combine b's lower nybble with a's high nybble
    or a, b
    ; And finally store it in RAM
    ld [buttons], a

...

SECTION "Important twiddles", WRAM0[$C000]
vblank_flag:
    db
buttons:
    db

Phew. That was a bit of a journey, but now I have the button state as a single byte. To help with reading the buttons, I’ll also define a few constants labeling the individual bits. (There are instructions for reading a particular bit by number, so I don’t need to mask a single bit out.)

1
2
3
4
5
6
7
8
9
; Constants
BUTTON_RIGHT  EQU 0
BUTTON_LEFT   EQU 1
BUTTON_UP     EQU 2
BUTTON_DOWN   EQU 3
BUTTON_A      EQU 4
BUTTON_B      EQU 5
BUTTON_START  EQU 6
BUTTON_SELECT EQU 7

Now to adjust the sprite position based on what directions are held down. Delete the old code and replace it with:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
    ; Set b/c to the y/x coordinates
    ld hl, oam_buffer
    ld b, [hl]
    inc hl
    ld c, [hl]

    ; This sets the z flag to match a particular bit in a
    bit BUTTON_LEFT, a
    ; If z, the bit is zero, so left isn't held down
    jr z, .skip_left
    ; Otherwise, left is held down, so decrement x
    dec c
.skip_left:

    ; The other three directions work the same way
    bit BUTTON_RIGHT, a
    jr z, .skip_right
    inc c
.skip_right:
    bit BUTTON_UP, a
    jr z, .skip_up
    dec b
.skip_up:
    bit BUTTON_DOWN, a
    jr z, .skip_down
    inc b
.skip_down:

    ; Finally, write the new coordinates back to the OAM
    ; buffer, which hl is still pointing into
    ld [hl], c
    dec hl
    ld [hl], b

Miraculously, Anise’s torso now moves around on command!

The half-cat is now moving according to button presses

Neat! But this still looks really, really, incredibly bad.

Aesthetics

It’s time to do something about this artwork.

First things first: I’m really tired of writing out colors by hand, in binary, so let’s fix that. In reality, I did this bit after adding better art, but doing it first is better for everyone.

I think I’ve mentioned before that rgbasm has (very, very rudimentary) support for macros, and this seems like a perfect use case for one. I’d like to be able to write colors out in typical rrggbb hex fashion, so I need to convert a 24-bit color to a 16-bit one.

1
2
3
4
5
6
dcolor: MACRO  ; $rrggbb -> gbc representation
_r = ((\1) & $ff0000) >> 16 >> 3
_g = ((\1) & $00ff00) >> 8  >> 3
_b = ((\1) & $0000ff) >> 0  >> 3
    dw (_r << 0) | (_g << 5) | (_b << 10)
    ENDM

This is going to need a whole paragraph of caveats.

A macro is contained between MACRO and ENDM. The assembler has a curious sort of universal assignment syntax, where even ephemeral constructs like macros are introduced by labels. Macros can take arguments, but they aren’t declared; they’re passed more like arguments to shell scripts, where the first argument is \1 and so forth. (There’s even a SHIFT command for accessing arguments beyond the ninth.) Also, passing strings to a macro is some kind of byzantine nightmare where you have to slap backslashes in just the right places and I will probably avoid doing it altogether if I can at all help it.

Oh, one other caveat: compile-time assignments like I have above must start in the first column. I believe this is because assignments are also labels, and labels have to start in the first column. It’s a bit weird and apparently rgbasm’s lexer is horrifying, but I’ll take it over writing my own assembler and stretching this project out any further.

Anyway, all of that lets me write dcolor $ff0044 somewhere and have it translated at compile time to the appropriate 16-bit value. (I used dcolor to parallel db and friends, but I’m strongly considering using CamelCase exclusively for macros? Guess it depends how heavily I use them.)

With that on hand, I can now doodle some little sprites in Aseprite and copy them in. This part is not especially interesting and involves a lot of squinting at zoomed-in sprites.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
SECTION "Sprites", ROM0
PALETTE_BG0:
    dcolor $80c870  ; light green
    dcolor $48b038  ; darker green
    dcolor $000000  ; unused
    dcolor $000000  ; unused
PALETTE_ANISE:
    dcolor $000000  ; TODO
    dcolor $204048
    dcolor $20b0b0
    dcolor $f8f8f8
GRASS_SPRITE:
    dw `00000000
    dw `00000000
    dw `01000100
    dw `01010100
    dw `00010000
    dw `00000000
    dw `00000000
    dw `00000000
EMPTY_SPRITE:
    dw `00000000
    dw `00000000
    dw `00000000
    dw `00000000
    dw `00000000
    dw `00000000
    dw `00000000
    dw `00000000
ANISE_SPRITE:
    ; ... I'll revisit this momentarily

Gorgeous. You may notice that I put the colors as data instead of inlining them in code, which incidentally makes the code for setting the palette vastly shorter as well:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
    ; Start setting the first color, and advance the internal
    ; pointer on every write
    ld a, %10000000
    ; BCPS = Background Color Palette Specification
    ldh [rBCPS], a

    ld hl, PALETTE_BG0
    REPT 8
    ld a, [hl+]
    ; Same, but Data
    ld [rBCPD], a
    ENDR

Loading sprites into VRAM also becomes a bit less of a mess:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
    ; Load some basic tiles
    ld hl, $8000

    ; Read the 16-byte empty sprite into tile 0
    ld bc, EMPTY_SPRITE
    REPT 16
    ld a, [bc]
    inc bc
    ld [hl+], a
    ENDR

    ; Read the grass sprite into tile 1, which immediately
    ; follows tile 0, so hl is already in the right place
    ld bc, GRASS_SPRITE
    REPT 16
    ld a, [bc]
    inc bc
    ld [hl+], a
    ENDR

Someday I should write an actual copy function, since at the moment, I’m using an alarming amount of space for pointlessly unrolled loops. Maybe later.

You may notice I now have two tiles, whereas before I was relying on filling the entire screen with one tile, tile 0. I want to dot the landscape with tile 1, which means writing a bit more to the actual background grid, which begins at $9800 and has one byte per tile.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
    ; Fill the screen buffer with a pattern of grass tiles,
    ; where every 2x2 block has a single grass at the top left.
    ; Note that the buffer is 32x32 tiles, and it ends at $9c00
    ld hl, $9800
.screen_fill_loop:
    ; Use tile 1 for every other tile in this row.  Note that
    ; REPTed part increments hl /twice/, thus skipping a tile
    ld a, $01
    REPT 16
    ld [hl+], a
    inc hl
    ENDR
    ; Skip an entire row of 32 tiles, which will remain empty.
    ; There is almost certainly a better way to do this, but I
    ; didn't do it.  (Hint: it's ld bc, $20; add hl, bc)
    REPT 32
    inc hl
    ENDR
    ; If we haven't reached $9c00 yet, continue looping
    ld a, h
    cp a, $9C
    jr c, .screen_fill_loop

Sorry for all these big blocks of code, but check out this payoff!

A very simple grassy background

POW! Gorgeous.

And hey, why stop there? With a little more pixel arting against a very reduced palette…

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
SPRITE_ANISE_FRONT_1:
    dw `00000111
    dw `00001222
    dw `00012222
    dw `00121222
    dw `00121122
    dw `00121111
    dw `00121122
    dw `00121312
    dw `00121313
    dw `00012132
    dw `00001211
    dw `00000123
    dw `00100123
    dw `00011133
    dw `00000131
    dw `00000010
SPRITE_ANISE_FRONT_2:
    dw `11100000
    dw `22210000
    dw `22221000
    dw `22212100
    dw `22112100
    dw `11112100
    dw `22112100
    dw `21312100
    dw `31312100
    dw `23121000
    dw `11210000
    dw `32100000
    dw `32100000
    dw `33100000
    dw `13100000
    dw `01000000

Yes, I am having trouble deciding on a naming convention.

This is now a 16×16 sprite, made out of two 8×16 parts. This post has enough code blocks as it is, and the changes to make this work are relatively minor copy/paste work, so the quick version is:

  1. Set the LCDC flag (bit 2, or LCDCF_OBJ16) that makes objects be 8×16. This mode uses pairs of tiles, so an object that uses either tile 0 or 1 will draw both of them, with tile 0 on top of tile 1.
  2. Extend the code that loads object tiles to load four instead.
  3. Define a second sprite that’s 8 pixels to the right of the first one.
  4. Remove the hard-coded object palette, and instead load the PALETTE_ANISE that I sneakily included above. This time the registers are called rOCPS and rOCPD.

Finally, extend the code that moves the sprite to also move the second half:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
    ; Finally, write the new coordinates back to the OAM
    ; buffer, which hl is still pointing into
    ld [hl], c
    dec hl
    ld [hl], b
    ; This bit is new: copy the x-coord into a so I can add 8
    ; to it, then store both coords into the second sprite's
    ; OAM data
    ld a, c
    add a, 8
    ; I could've written this the other way around, but I did
    ; not, I guess because this structure mirrors the above?
    ld hl, oam_buffer + 5
    ld [hl], a
    dec hl
    ld [hl], b

Cross my fingers, and…

A little cat sprite atop the grassy background

Hey hey hey! That finally looks like something!

To be continued

It was a surprisingly long journey, but this brings us more or less up to commit 313a3e, which happens to be the first commit I made a release of! It’s been more than a week, so you can grab it on Patreon or GitHub. I strongly recommend playing it with a release of mGBA prior to 0.7, for… reasons that will become clear next time.

Next time: I’ll take a breather and clean up a few things.

Cheezball Rising: Drawing a sprite

Post Syndicated from Eevee original https://eev.ee/blog/2018/06/21/cheezball-rising-drawing-a-sprite/

This is a series about Star Anise Chronicles: Cheezball Rising, an expansive adventure game about my cat for the Game Boy Color. Follow along as I struggle to make something with this bleeding-edge console!

source codeprebuilt ROMs (a week early for $4) • works best with mGBA

In this issue, I figure out how to draw a sprite. This part was hard.

Previously: figuring out how to put literally anything on the goddamn screen.

Recap

Welcome back! I’ve started cobbling together a Pygments lexer for RGBDS’s assembly flavor, so hopefully the code blocks are more readable, and will become moreso over time.

When I left off last time, I had… um… this.

Vertical stripes of red, green, blue, and white

This is all on the background layer, which I mentioned before is a fixed grid of 8×8 tiles.

For anything that moves around freely, like the player, I need to use the object layer. So that’s an obvious place to go next.

Now, if you remember, I can define tiles by just writing to video RAM, and I define palettes with a goofy system involving writing them one byte at a time to the same magic address. You might expect defining objects to do some third completely different thing, and you’d be right!

Defining an object

Objects are defined in their own little chunk of RAM called OAM, for object attribute memory. They’re also made up of tiles, but each tile can be positioned at an arbitrary point on the screen.

OAM starts at $fe00 and each object takes four bytes — the y-coordinate, the x-coordinate, the tile number, and some flags — for a total of 160 bytes. There are some curiosities, like how the top left of the screen is (8, 10) rather than (0, 0), but I’ll figure out what’s up with that later. (I suppose if zeroes meant the upper left corner, there’d be a whole stack of tile 0 there all the time.)

Here’s the fun part: I can’t write directly to OAM? I guess??? Come to think of it, I don’t think the manual explicitly says I can’t, but it’s strongly implied. Hmm. I’ll look into that. But I didn’t at the time, so I’ll continue under the assumption that the following nonsense is necessary.

Because I “can’t” write directly, I need to use some shenanigans. First, I need something to write! This is an Anise game, so let’s go for Anise.

I’m on my laptop at this point without access to the source code for the LÖVE Anise game I started, so I have to rustle up a screenshot I took.

Cropped screenshot of Star Anise and some critters, all pixel art

Wait a second.

Even on the Game Boy Color, tiles are defined with two bits per pixel. That means an 8×8 tile has a maximum of four colors. For objects, the first color is transparent, so I really have three colors — which is exactly why most Game Boy Color protagonists have a main color, an outline/shadow color, and a highlight color.

Let’s check out that Anise in more detail.

Star Anise at 8×

Hm yes okay that’s more than three colors. I guess I’m going to need to draw some new sprites from scratch, somehow.

In the meantime, I optimistically notice that Star Anise’s body only uses three colors, and it’s 8×7! I could make a tile out of that! I painstakingly copy the pixels into a block of those backticks, which you can kinda see is his body if you squint a bit:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
SECTION "Sprites", ROM0
ANISE_SPRITE:
    dw `00000000
    dw `00001333
    dw `00001323
    dw `10001233
    dw `01001333
    dw `00113332
    dw `00003002
    dw `00003002

The dw notation isn’t an opcode; it tells the assembler to put two literal bytes of data in the final ROM. A word of data. (Each row of a tile is two bytes, remember.)

If you think about this too hard, you start to realize that both the data and code are just bytes, everything is arbitrary, and true meaning is found only in the way we perceive things rather than in the things themselves.

Note I didn’t specify an exact address for this section, so the linker will figure out somewhere to put it and make sure all the labels are right at the end.

Now I load this into tilespace, back in my main code:

1
2
3
4
5
6
7
8
    ; Define an object
    ld hl, $8800
    ld bc, ANISE_SPRITE
    REPT 16
    ld a, [bc]
    ld [hl+], a
    inc bc
    ENDR

This copies 16 bytes, starting from the ANISE_SPRITE label, to $8800.


Why $8800, not $8000? I’m so glad you asked!

There are actually three blocks of tile space, each with enough room for 128 tiles: one at $8000, one at $8800, and one at $9000. Object tiles always use the $8000 block followed by the $8800 block, whereas background tiles can use either $8000 + $8800 or $9000 + $8800. By default, background tiles use $8000 + $8800.

All of which is to say that I got very confused reading the manual (which spends like five pages explaining the above paragraph) and put the object tiles in the wrong place. Whoops. It’s fine; this just ends up being tile 128.

In my partial defense, looking at it now, I see the manual is wrong! Bit 4 of the LCD controller register ($ff40) controls whether the background uses tiles from $8000 + $8800 (1) or $9000 + $8800 (0). The manual says that this register defaults to $83, which has bit 4 off, suggesting that background tiles use $9000 + $8800 (i.e. start at $8800), but disassembly of the boot ROM shows that it actually defaults to $91, which has bit 4 on. Thanks a lot, Nintendo!

That was quite a diversion. Here’s a chart of where the dang tiles live. Note that the block at $8800 is always shared between objects and background tiles. Oh, and on the Game Boy Color, all three blocks are twice as big thanks to the magic of banking. I’ll get to banking… much later.

1
2
3
4
5
                            bit 4 ON (default)  bit 4 OFF
                            ------------------  ---------
$8000   obj tiles 0-127     bg tiles 0-127
$8800   obj tiles 128-255   bg tiles 128-255    bg tiles 128-255
$9000                                           bg tiles 0-127

Hokay. What else? I’m going to need a palette for this, and I don’t want to use that gaudy background palette. Actually, I can’t — the background and object layers have two completely separate sets of palettes.

Writing an object palette is exactly the same as writing a background palette, except with different registers.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
    ; This should look pretty familiar
    ld a, %10000000
    ld [$ff6a], a

    ld bc, %0000000000000000  ; transparent
    ld a, c
    ld [$ff6b], a
    ld a, b
    ld [$ff6b], a
    ld bc, %0010110100100101  ; dark
    ld a, c
    ld [$ff6b], a
    ld a, b
    ld [$ff6b], a
    ld bc, %0100000111001101  ; med
    ld a, c
    ld [$ff6b], a
    ld a, b
    ld [$ff6b], a
    ld bc, %0100001000010001  ; white
    ld a, c
    ld [$ff6b], a
    ld a, b
    ld [$ff6b], a

Riveting!

I wrote out those colors by hand. The original dark color, for example, was #264a59. That uses eight bits per channel, but the Game Boy Color only supports five (a factor of 8 difference), so first I rounded each channel to the nearest 8 and got #284858. Swap the channels to get 58 48 28 and convert to binary (sans the trailing zeroes) to get 01011 01001 00101.

Note to self: probably write a macro or whatever so I can define colors like a goddamn human being. Also why am I not putting the colors in a ROM section too?

Almost there. I still need to write out those four bytes that specify the tile and where it goes. I can’t actually write them to OAM yet, so I need some scratch space in regular RAMworking RAM.

1
2
3
SECTION "OAM Buffer", WRAM0[$C100]
oam_buffer:
    ds 4 * 40

The ds notation is another “data” variant, except it can take a size and reserves space for a whole string of data. Note that I didn’t put any actual data here — this section is in RAM, which only exists while the game is running, so there’d be nowhere to put data.

Also note that I gave an explicit address this time. The buffer has to start at an address ending in 00, for reasons that will become clear momentarily. The space from $c000 to $dfff is available as working RAM, and I chose $c100 for… reasons that will also become clear momentarily.

Now to write four bytes to it at runtime:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
    ; Put an object on the screen
    ld hl, oam_buffer
    ; y-coord
    ld a, 64
    ld [hl+], a
    ; x-coord
    ld [hl+], a
    ; tile index
    ld a, 128
    ld [hl+], a
    ; attributes, including palette, which are all zero
    ld a, %00000000
    ld [hl+], a

(I tried writing directly to OAM on my first attempt. Nothing happened! Very exciting.)

But how to get this into OAM so it’ll actually show on-screen? For that, I need to do a DMA transfer.

DMA

DMA, or direct memory access, is one of those things the Game Boy programming manual seems to think everyone is already familiar with. It refers generally to features that allow some other hardware to access memory, without going through the CPU. In the case of the Game Boy, it’s used to copy data from working RAM to OAM. Only to OAM. It’s very specific.

Performing a DMA transfer is super easy! I write the high byte of the source address to the DMA register ($ff46), and then some magic happens, and 160 bytes from the source address appear in OAM. In other words:

1
2
3
    ld a, $c1       ; copy from $c100
    ld [$ff46], a   ; perform DMA transfer
    ; now $c000 through $c09f have been copied into OAM!

It’s almost too good to be true! And it is. There are some wrinkles.

First, the transfer takes some time, during which I almost certainly don’t want to be doing anything else.

Second, during the transfer, the CPU can only read from “high RAM” — $ff80 and higher. Wait, uh oh.

The usual workaround here is to copy a very short function into high RAM to perform the actual transfer and wait for it to finish, then call that instead of starting a transfer directly. Well, that sounds like a pain, so I break my rule of accounting for every byte and find someone else who’s done it. Conveniently enough, that post is by the author of the small template project I’ve been glancing at.

I end up with something like the following.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
    ; Copy the little DMA routine into high RAM
    ld bc, DMA_BYTECODE
    ld hl, $ff80
    ; DMA routine is 13 bytes long
    REPT 13
    ld a, [bc]
    inc bc
    ld [hl+], a
    ENDR

; ...

SECTION "DMA Bytecode", ROM0
DMA_BYTECODE:
    db $F5, $3E, $C1, $EA, $46, $FF, $3E, $28, $3D, $20, $FD, $F1, $D9

That’s compiled assembly, written inline as bytes. Oh boy. The original code looks like:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
    ; start the transfer, as shown above
    ld a, $c1
    ld [$ff46], a

    ; wait 160 cycles/microseconds, the time it takes for the
    ; transfer to finish; this works because 'dec' is 1 cycle
    ; and 'jr' is 3, for 4 cycles done 40 times
    ld      a, 40
loop:
    dec     a
    jr      nz, loop

    ; return
    ret

Now you can see why I used $c100 for my OAM buffer: because it’s the address this person used.

(Hm, the opcode reference I usually use seems to have all the timings multiplied by a factor of 4 without comment? Odd. The rgbds reference is correct.)

(Also, here’s a fun fact: the stack starts at $fffe and grows backwards. If it grows too big, the very first thing it’ll overwrite is this DMA routine! I bet that’ll have some fun effects.)

At this point I have a thought. (Okay, I had the thought a bit later, but it works better narratively if I have it now.) I’ve already demonstrated that the line between code and data is a bit fuzzy here. So why does this code need to be pre-assembled?

And a similar thought: why is the length hardcoded? Surely, we can do a little better. What if we shuffle things around a bit…

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
SECTION "init", ROM0[$0100]
    nop
    ; Jump to a named label instead of an address
    jp main

SECTION "main", ROM0[$0150]
; DMA copy routine, copied into high RAM at startup.
; Never actually called where it is.
dma_copy:
    ld a, $c1
    ld [$ff46], a
    ld a, 40
.loop:
    dec a
    jr nz, .loop
    ret
dma_copy_end:
    nop

main:
    ; ... all previous code is here now ...

    ; Copy the little DMA routine into high RAM
    ld bc, dma_copy
    ld hl, $ff80
    ; DMA routine is 13 bytes long
    REPT dma_copy_end - dma_copy
    ld a, [bc]
    inc bc
    ld [hl+], a
    ENDR

This is very similar to what I just had, except that the code is left as code, and its length is computed by having another label at the end — so I’m free to edit it later if I want to. It all ends up as bytes in the ROM, so the code ends up exactly the same as writing out the bytes with db. Come to think of it, I don’t even need to hardcode the $c1 there; I could replace it with oam_buffer >> 8 and avoid repeating myself.

(I put the code at $0150 because rgbasm is very picky about subtracting labels, and will only do it if they both have fixed positions. These two labels would be the same distance apart no matter where I put the section, but I guess rgbasm isn’t smart enough to realize that.)

I’m actually surprised that the author of the above post didn’t think to do this? Maybe it’s dirty even by assembly standards.

Timing, vblank, and some cool trickery

Okay, so, as I was writing that last section, I got really curious about whether and when I’m actually allowed to write to OAM. Or tile RAM, for that matter.

I found/consulted the Game Boy dev wiki, and the rules match what’s in the manual, albeit with a chart that makes things a little more clear.

My understanding is as follows. The LCD draws the screen one row of pixels at a time, and each row has the following steps:

  1. Look through OAM to see if any sprites are on this row. OAM is inaccessible to the CPU.

  2. Draw the row. OAM, VRAM, and palettes are all inaccessible.

  3. Finish the row and continue on to the beginning of the next row. This takes a nonzero amount of time, called the horizontal blanking period, during which the CPU can access everything freely.

Once the LCD reaches the bottom, it continues to “draw” a number of faux rows below the bottom of the visible screen (vertical blanking), and the CPU can again do whatever it wants. Eventually it returns to the top-left corner to draw again, concluding a single frame. The entire process happens 59.7 times per second.

There’s one exception: DMA transfers can happen any time, but the LCD will simply not draw sprites during the transfer.

So I probably shouldn’t be writing to tiles and palettes willy-nilly. I suspect I got away with it because it happened in that first OAM-searching stage… and/or because I did it on emulators which are a bit more flexible than the original hardware.

In fact…

Same screenshot as above, but the first row of pixels is corrupt

I took this screenshot by loading the ROM I have so far, pausing it, resetting it, and then advancing a single frame. This is the very first frame my game shows. If you look closely at the first row of pixels, you can see they’re actually corrupt — they’re being drawn before I’ve set up the palette! You can even see each palette entry taking effect along the row.

This is very cool. It also means my current code would not work at all on actual hardware. I should probably just turn the screen off while I’m doing setup like this.

It’s interesting that only OAM gets a special workaround in the form of a DMA transfer — I imagine because sprites move around much more often than the tileset changes — but having the LCD stop drawing sprites in the meantime is quite a limitation. Surely, you’d only want to do a DMA transfer during vblank anyway? It is much faster than copying by hand, so I’ll still take it.

All of this is to say: I’m gonna need to care about vblanks.


Incidentally, the presence of hblank is very cool and can be used for a number of neat effects, especially when combined with the Game Boy’s ability to call back into user code when the LCD reaches a specific row:

  • The GBC Zelda games use it for map scrolling. The status bar at the top is in one of the two background maps, and as soon as that finishes drawing, the game switches to the other one, which contains the world.

  • Those same games also use it for a horizontal wavy effect, both when warping around and when underwater — all they need to do is change the background layer’s x offset during each hblank!

  • The wiki points out that OAM could be written to in the middle of a screen update, thus bypassing the 40-object restriction: draw 40 objects on the top half of the screen, swap out OAM midway, and then the LCD will draw a different 40 on the bottom half!

  • I imagine you could also change palettes midway through a redraw and exceed the usual limit of 56 colors on screen at a time! No telling whether this sort of trick would work on an emulator, though.

I am very excited at the prospects here.

I’m also slightly terrified. I have a fixed amount of time between frames, and with the LCD as separate hardware, there’s no such thing as a slow frame. If I don’t finish, things go bad. And that time is measured in instructions — an ld always takes the same number of cycles! There’s no faster computer or reducing GC pressure. There’s just me. Yikes.

Back to drawing a sprite

I haven’t had a single new screenshot this entire post! This is ridiculous. All I want is to draw a thing to the screen.

I have some data in my OAM buffer. I have DMA set up. All I should need to do now is start a transfer.

1
    call $ff80

And… nothing. mGBA’s memory viewer confirms everything’s in the right place, but nothing’s on the screen.

Whoops! Remember that LCD controller register, and how it defaults to $91? Well, bit 1 is whether to show objects at all, and it defaults to off. So let’s fix that.

1
2
    ld a, %10010011  ; $91 plus bit 2
    ld [$ff40], a
The same gaudy background, but now with a partial Anise sprite on top

SUCCESS!

It doesn’t look like much, but it took a lot of flailing to get here, and I was overjoyed when I first saw it. The rest should be a breeze! Right?

To be continued

That doesn’t even get us all the way through commit 1b17c7, but this is already more than enough.

Next time: input, and moderately less eye-searing art!

Cheezball Rising: A new Game Boy Color game

Post Syndicated from Eevee original https://eev.ee/blog/2018/06/19/cheezball-rising-a-new-game-boy-color-game/

This is a series about Star Anise Chronicles: Cheezball Rising, an expansive adventure game about my cat for the Game Boy Color. Follow along as I struggle to make something with this bleeding-edge console!

source codeprebuilt ROMs (a week early for $4) • works best with mGBA

In this issue, I figure out how to put literally anything on the goddamn screen, then add a splash of color.

The plan

I’m making a Game Boy Color game!

I have no— okay, not much idea what I’m doing, so I’m going to document my progress as I try to forge a 90s handheld game out of nothing.

I do usually try to keep tech stuff accessible, but this is going to get so arcane that that might be a fool’s errand. Think of this as less of an extended tutorial, more of a long-form Twitter.

Also, I’ll be posting regular builds on Patreon for $4 supporters, which will be available a week later for everyone else. I imagine they’ll generally stay in lockstep with the posts, unless I fall behind on the writing part. But when has that ever happened?

Your very own gamedev legend is about to unfold! A world of dreams and adventures with gbz80 assembly awaits! Let’s go!

Prerequisites

First things first. I have a teeny bit of experience with Game Boy hacking, so I know I need:

  • An emulator. I have no way to run arbitrary code on an actual Game Boy Color, after all. I like mGBA, which strives for accuracy and has some debug tools built in.

    There’s already a serious pitfall here: emulators are generally designed to run games that would work correctly on the actual hardware, but they won’t necessarily reject games that wouldn’t work on actual hardware. In other words, something that works in an emulator might still not work on a real GBC. I would of course prefer that this game work on the actual console it’s built for, but I’ll worry about that later.

  • An assembler, which can build Game Boy assembly code into a ROM. I pretty much wrote one of these myself already for the Pokémon shenanigans, but let’s go with something a little more robust here. I’m using RGBDS, which has a couple nice features like macros and a separate linking step. It compiles super easily, too.

    I also hunted down a vim syntax file, uh, somewhere. I can’t remember which one it was now, and it’s kind of glitchy anyway.

  • Some documentation. I don’t know exactly how this surfaced, but the actual official Game Boy programming manual is on archive.org. It glosses over some things and assumes some existing low-level knowledge, but for the most part it’s a very solid reference.

For everything else, there’s Google, and also the curated awesome-gbdev list of resources.

That list includes several skeleton projects for getting started, but I’m not going to use them. I want to be able to account for every byte of whatever I create. I will, however, refer to them if I get stuck early on. (Spoilers: I get stuck early on.)

And that’s it! The rest is up to me.

Making nothing from nothing

Might as well start with a Makefile. The rgbds root documentation leads me to the following incantation:

1
2
3
4
all:
        rgbasm -o main.o main.rgbasm
        rgblink -o gamegirl.gb main.o
        rgbfix -v -p 0 gamegirl.gb

(I, uh, named this project “gamegirl” before I figured out what it was going to be. It’s a sort of witticism, you see.)

This works basically like every C compiler under the sun, as you might expect: every source file compiles to an object file, then a linker bundles all the object files into a ROM. If I only change one source file, I only have to rebuild one object file.

Of course, this Makefile is terrible garbage and will rebuild the entire project unconditionally every time, but at the moment that takes a fraction of a second so I don’t care.

The extra rgbfix step is new, though — it adds the Nintendo logo (the one you see when you start up a Game Boy) to the header at the beginning of the ROM. Without this, the console will assume the cartridge is dirty or missing or otherwise unreadable, and will refuse to do anything at all. (I could also bake the logo into the source itself, but given that it’s just a fixed block of bytes and rgbfix is bundled with the assembler, I see no reason to bother with that.)

All I need now is a source file, main.rgbasm, which I populate with:

1

Nothing! I don’t know what I expect from this, but I’m curious to see what comes out. And what comes out is a working ROM!

A completely blank screen

Maybe “working” is a strong choice of word, given that it doesn’t actually do anything.

Doing something

It would be fantastic to put something on the screen. This turned out to be harder than expected.

First attempt. I know that the Game Boy starts running code at $0150, immediately after the end of the header. So I’ll put some code there.

A brief Game Boy graphics primer: there are two layers, the background and objects. (There’s also a third layer, the window, which I don’t entirely understand yet.) The background is a grid of 8×8 tiles, two bits per pixel, for a total of four shades of gray. Objects can move around freely, but they lose color 0 to transparency, so they can only use three colors.

There are lots more interesting details and restrictions, which I will think about more later.

Drawing objects is complicated, and all I want to do right now is get something. I’m pretty sure the background defaults to showing all tile 0, so I’ll try replacing tile 0 with a gradient and see what happens.

Tiles are 8×8 and two bits per pixel, which means each row takes two bytes, and the whole tile is 16 bytes. Tiles are defined in one big contiguous block starting at $8000 — or, maybe $8800, sometimes — so all I need to do is:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
SECTION "main", ROM0[$0150]
    ld hl, $8000
    ld a, %00011011
    REPT 16
    ld [hl+], a
    ENDR

_halt:
    ; Do nothing, forever
    halt
    nop
    jr _halt

If you are not familiar with assembly, this series is going to be a wild ride. But here’s a very very brief primer.

Assembly language — really, an assembly language — is little more than a set of human-readable names for the primitive operations a CPU knows how to do. And those operations, by and large, consist of moving bytes around. The names tend to be very short, because you end up typing them a lot.

Most of the work is done in registers, which are a handful of spaces for storing bytes right on the CPU. At this level, RAM is relatively slow — it’s further away, outside the chip — so you want to do as much work as possible in registers. Indeed, most operations can only be done on registers, so there’s a lot of fetching stuff from RAM and operating on it and then putting it back in RAM.

The Game Boy CPU, a modified Z80, has eight byte-sized registers. They’re often referred to in pairs, because they can be paired up to make a 16-bit values (giving you access to a full 64KB address space). And they are: af, bc, de, hl.

The af pair is special. The f register is used for flags, such as whether the last instruction caused an overflow, so it’s not generally touched directly. The a register is called the accumulator and is most commonly used for math operations — in fact, a lot of math operations can only be done on a. The hl register is most often used for addresses, and there are a couple instructions specific to hl that are convenient for memory access. (The h and l even refer to the high and low byte of an address.) The other two pairs aren’t especially noteworthy.

Also! Not every address is actually RAM; the address space ($0000 through $ffff) is carved into several distinct areas, which we will see as I go along. $8000 is the beginning of display RAM, which the screen reads from asynchronously. Also, a lot of addresses above $ff00 (also called “registers”) are special and control hardware in some way, or even perform some action when written to.

With that in mind, here’s the above code with explanatory comments:

TODO need to change this to write a single byte

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
; This is a directive for the assembler to put the following
; code at $0150 in the final ROM.
SECTION "main", ROM0[$0150]
    ; Put the hex value $8000 into registers hl.  Really, that
    ; means put $80 into h and $00 into l.
    ld hl, $8000

    ; Put this binary value into registers a.
    ; It's just 0 1 2 3, a color gradient.
    ld a, %00011011

    ; This is actually a macro this particular assembler
    ; understands, which will repeat the following code 16
    ; times, exactly as if I'd copy-pasted it.
    REPT 16

    ; The brackets (sometimes written as parens) mean to use hl
    ; as a position in RAM, rather than operating on hl itself.
    ; So this copies a into the position in RAM given by
    ; hl (initially $8000), and the + adds 1 to hl afterwards.
    ; This is one reason hl is nice for storing addresses: the +
    ; variant is handy for writing a sequence of bytes to RAM,
    ; and it only exists for hl.
    ld [hl+], a

    ; End the REPT block
    ENDR

; This is a label, used to refer to some position in the code.
; It only exists in the source file.
_halt:
    ; Stop all CPU activity until there's an interrupt.  I
    ; haven't turned any interrupts on, so this stops forever.
    halt

    ; The Game Boy hardware has a bug where, under rare and
    ; unspecified conditions, the instruction after a halt will
    ; be skipped.  So every halt should be followed by a nop,
    ; "no operation", which does nothing.
    nop

    ; This jumps back up to the label.  It's short for "jump
    ; relative", and will end up as an instruction saying
    ; something like "jump backwards five bytes", or however far
    ; back _halt is.  (Different instructions can be different
    ; lengths.)
    jr _halt

Okay! Glad you’re all caught up. The rgbds documentation includes a list of all the available operations (as well as assembler syntax), and once you get used to the short names, I also like this very compact chart of all the instructions and how they compile to machine code. (Note that that chart spells [hl+] as (HLI), for “increment” — the human-readable names are somewhat arbitrary and can sometimes vary between assemblers.)

Now, let’s see what this does!


A completely blank screen, still

Wow! It’s… still nothing. Hang on.

If I open the debugger and hit Break, I find out that the CPU is at address $0120 — before my code — and is on an instruction DD. What’s DD? Well, according to this convenient chart, it’s… nothing. That’s not an instruction.

Hmm.

Problem solving

Maybe it’s time to look at one of those skeleton projects after all. I crack open the smallest one, gb-template, and it seems to be doing the same thing: its code istarts at $0150.

It takes me a bit to realize my mistake here. Practically every Game Boy game starts its code at $0150, but that’s not what the actual hardware specifies. The real start point is $0100, which is immediately before the header! There are only four bytes before the header, just enough for… a jump instruction.

Okay! No problem.

1
2
3
SECTION "entry point", ROM0[$0100]
    nop
    jp $0150

Why the nop? I have no idea, but all of these boilerplate projects do it.

Black screen with repeating columns of white

Uhh.

Well, that’s weird. Not only is the result black and white when I definitely used all four shades, but the whites aren’t even next to each other. (I also had a strange effect where the screen reverted to all white after a few seconds, but can’t reproduce it now; it was fixed by the same steps, though, so it may have been a quirk of a particular mGBA build.)

I’ll save you my head-scratching. I made two mistakes here. Arguably, three!

First: believe it or not, I have to specify the palette. Even in original uncolored Game Boy mode! I can see how that’s nice for doing simple fade effects or flashing colors, but I didn’t suspect it would be necessary. The monochrome palette lives at $ff47 (one of those special high addresses), so I do this before anything else:

1
2
    ld a, %11100100         ; 3 2 1 0
    ld [$ff47], a

I should really give names to some of these special addresses, but for now I’m more interested in something that works than something that’s nice to read.

Second: I specified the colors wrong. I assumed that eight pixels would fit into two bytes as AaBbCcDd EeFfGgHh, perhaps with some rearrangement, but a closer look at Nintendo’s manual reveals that they need to be ABCDEFGH abcdefgh, with the two bits for each pixel split across each byte! Wild.

Handily, rgbds has syntax for writing out pixel values directly: a backtick followed by eight of 0, 1, 2, and 3. I just have to change my code a bit to write two bytes, eight times each. By putting a 16-bit value in a register pair like bc, I can read its high and low bytes out individually via the b and c registers.

1
2
3
4
5
6
7
8
    ld hl, $8000
    ld bc, `00112233
    REPT 8
    ld a, b
    ld [hl+], a
    ld a, c
    ld [hl+], a
    ENDR

Third: strictly speaking, I don’t think I should be writing to $8000 while the screen is on, because the screen may be trying to read from it at the same time. It does happen to work in this emulator, but I have no idea whether it would work on actual hardware. I’m not going to worry too much about this test code; most likely, tile loading will happen all in one place in the real game, and I can figure out any issues then.

This is one of those places where the manual is oddly vague. It dedicates two whole pages to diagrams of how sprites are drawn when they overlap, yet when I can write to display RAM is left implicit.

Well, whatever. It works on my machine.

Stripes of varying shades of gray

Success! I made a thing for the Game Boy.

Ah, but what I wanted was a thing for the Game Boy Color. That shouldn’t be too much harder.

Now in Technicolor

First I update my Makefile to pass the -C flag to rgbfix. That tells it to set a flag in the ROM header to indicate that this game is only intended for the Game Boy Color, and won’t work on the original Game Boy. (In order to pass Nintendo certification, I’ll need an error screen when the game is run on a non-Color Game Boy, but that can come later. Also, I don’t actually know how to do that.)

Oh, and I’ll change the file extension from .gb to .gbc. And while I’m in here, I might as well repeat myself slightly less in this bad, bad Makefile.

1
2
3
4
5
6
7
8
TARGET := gamegirl.gbc

all: $(TARGET)

$(TARGET):
        rgbasm -o main.o main.rgbasm
        rgblink -o $(TARGET) main.o
        rgbfix -C -v -p 0 $(TARGET)

I think := is the one I want, right? Christ, who can remember how this syntax works.

Next I need to define a palette. Again, everything defaults to palette zero, so I’ll update that and not have to worry about specifying a palette for every tile.

This part is a bit weird. Unlike tiles, there’s not a block of addresses somewhere that contains all the palettes. Instead, I have to write the palette to a single address one byte at a time, and the CPU will put it… um… somewhere.

(I think this is because the entire address space was already carved up for the original Game Boy, and they just didn’t have room to expose palettes, but they still had a few spare high addresses they could use for new registers.)

Two registers are involved here. The first, $ff68, specifies which palette I’m writing to. It has a bunch of parts, but since I’m writing to the first color of palette zero, I can leave it all zeroes. The one exception is the high bit, which I’ll explain in just a moment.

1
2
    ld a, %10000000
    ld [$ff68], a

The other, $ff69, does the actual writing. Each color in a palette is two bytes, and a palette contains four colors, so I need to write eight bytes to this same address. The high bit in $ff68 is helpful here: it means that every time I write to $ff69, it should increment its internal position by one. This is kind of like the [hl+] I used above: after every write, the address increases, so I can just write all the data in sequence.

But first I need some colors! Game Boy Color colors are RGB555, which means each color is five bits (0–31) and a full color fits in two bytes: 0bbbbbgg gggrrrrr.

(I got this backwards initially and thought the left bits were red and the right bits were blue.)

Thus, I present, palette loading by hand. Like before, I put the 16-bit color in bc and then write out the contents of b and c. (Before, the backtick syntax put the bytes in the right order; colors are little-endian, hence why I write c before b.)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
    ld bc, %0111110000000000  ; blue
    ld a, c
    ld [$ff69], a
    ld a, b
    ld [$ff69], a
    ld bc, %0000001111100000  ; green
    ld a, c
    ld [$ff69], a
    ld a, b
    ld [$ff69], a
    ld bc, %0000000000011111  ; red
    ld a, c
    ld [$ff69], a
    ld a, b
    ld [$ff69], a
    ld bc, %0111111111111111  ; white
    ld a, c
    ld [$ff69], a
    ld a, b
    ld [$ff69], a

Rebuild, and:

Same as before, but now the stripes are colored

What a glorious eyesore!

To be continued

That brings us up to commit 212344 and works as a good stopping point.

Next time: sprites! Maybe even some real art?

AWS Online Tech Talks – June 2018

Post Syndicated from Devin Watson original https://aws.amazon.com/blogs/aws/aws-online-tech-talks-june-2018/

AWS Online Tech Talks – June 2018

Join us this month to learn about AWS services and solutions. New this month, we have a fireside chat with the GM of Amazon WorkSpaces and our 2nd episode of the “How to re:Invent” series. We’ll also cover best practices, deep dives, use cases and more! Join us and register today!

Note – All sessions are free and in Pacific Time.

Tech talks featured this month:

 

Analytics & Big Data

June 18, 2018 | 11:00 AM – 11:45 AM PTGet Started with Real-Time Streaming Data in Under 5 Minutes – Learn how to use Amazon Kinesis to capture, store, and analyze streaming data in real-time including IoT device data, VPC flow logs, and clickstream data.
June 20, 2018 | 11:00 AM – 11:45 AM PT – Insights For Everyone – Deploying Data across your Organization – Learn how to deploy data at scale using AWS Analytics and QuickSight’s new reader role and usage based pricing.

 

AWS re:Invent
June 13, 2018 | 05:00 PM – 05:30 PM PTEpisode 2: AWS re:Invent Breakout Content Secret Sauce – Hear from one of our own AWS content experts as we dive deep into the re:Invent content strategy and how we maintain a high bar.
Compute

June 25, 2018 | 01:00 PM – 01:45 PM PTAccelerating Containerized Workloads with Amazon EC2 Spot Instances – Learn how to efficiently deploy containerized workloads and easily manage clusters at any scale at a fraction of the cost with Spot Instances.

June 26, 2018 | 01:00 PM – 01:45 PM PTEnsuring Your Windows Server Workloads Are Well-Architected – Get the benefits, best practices and tools on running your Microsoft Workloads on AWS leveraging a well-architected approach.

 

Containers
June 25, 2018 | 09:00 AM – 09:45 AM PTRunning Kubernetes on AWS – Learn about the basics of running Kubernetes on AWS including how setup masters, networking, security, and add auto-scaling to your cluster.

 

Databases

June 18, 2018 | 01:00 PM – 01:45 PM PTOracle to Amazon Aurora Migration, Step by Step – Learn how to migrate your Oracle database to Amazon Aurora.
DevOps

June 20, 2018 | 09:00 AM – 09:45 AM PTSet Up a CI/CD Pipeline for Deploying Containers Using the AWS Developer Tools – Learn how to set up a CI/CD pipeline for deploying containers using the AWS Developer Tools.

 

Enterprise & Hybrid
June 18, 2018 | 09:00 AM – 09:45 AM PTDe-risking Enterprise Migration with AWS Managed Services – Learn how enterprise customers are de-risking cloud adoption with AWS Managed Services.

June 19, 2018 | 11:00 AM – 11:45 AM PTLaunch AWS Faster using Automated Landing Zones – Learn how the AWS Landing Zone can automate the set up of best practice baselines when setting up new

 

AWS Environments

June 21, 2018 | 11:00 AM – 11:45 AM PTLeading Your Team Through a Cloud Transformation – Learn how you can help lead your organization through a cloud transformation.

June 21, 2018 | 01:00 PM – 01:45 PM PTEnabling New Retail Customer Experiences with Big Data – Learn how AWS can help retailers realize actual value from their big data and deliver on differentiated retail customer experiences.

June 28, 2018 | 01:00 PM – 01:45 PM PTFireside Chat: End User Collaboration on AWS – Learn how End User Compute services can help you deliver access to desktops and applications anywhere, anytime, using any device.
IoT

June 27, 2018 | 11:00 AM – 11:45 AM PTAWS IoT in the Connected Home – Learn how to use AWS IoT to build innovative Connected Home products.

 

Machine Learning

June 19, 2018 | 09:00 AM – 09:45 AM PTIntegrating Amazon SageMaker into your Enterprise – Learn how to integrate Amazon SageMaker and other AWS Services within an Enterprise environment.

June 21, 2018 | 09:00 AM – 09:45 AM PTBuilding Text Analytics Applications on AWS using Amazon Comprehend – Learn how you can unlock the value of your unstructured data with NLP-based text analytics.

 

Management Tools

June 20, 2018 | 01:00 PM – 01:45 PM PTOptimizing Application Performance and Costs with Auto Scaling – Learn how selecting the right scaling option can help optimize application performance and costs.

 

Mobile
June 25, 2018 | 11:00 AM – 11:45 AM PTDrive User Engagement with Amazon Pinpoint – Learn how Amazon Pinpoint simplifies and streamlines effective user engagement.

 

Security, Identity & Compliance

June 26, 2018 | 09:00 AM – 09:45 AM PTUnderstanding AWS Secrets Manager – Learn how AWS Secrets Manager helps you rotate and manage access to secrets centrally.
June 28, 2018 | 09:00 AM – 09:45 AM PTUsing Amazon Inspector to Discover Potential Security Issues – See how Amazon Inspector can be used to discover security issues of your instances.

 

Serverless

June 19, 2018 | 01:00 PM – 01:45 PM PTProductionize Serverless Application Building and Deployments with AWS SAM – Learn expert tips and techniques for building and deploying serverless applications at scale with AWS SAM.

 

Storage

June 26, 2018 | 11:00 AM – 11:45 AM PTDeep Dive: Hybrid Cloud Storage with AWS Storage Gateway – Learn how you can reduce your on-premises infrastructure by using the AWS Storage Gateway to connecting your applications to the scalable and reliable AWS storage services.
June 27, 2018 | 01:00 PM – 01:45 PM PTChanging the Game: Extending Compute Capabilities to the Edge – Discover how to change the game for IIoT and edge analytics applications with AWS Snowball Edge plus enhanced Compute instances.
June 28, 2018 | 11:00 AM – 11:45 AM PTBig Data and Analytics Workloads on Amazon EFS – Get best practices and deployment advice for running big data and analytics workloads on Amazon EFS.

When Joe Public Becomes a Commercial Pirate, a Little Knowledge is Dangerous

Post Syndicated from Andy original https://torrentfreak.com/joe-public-becomes-commercial-pirate-little-knowledge-dangerous-180603/

Back in March and just a few hours before the Anthony Joshua v Joseph Parker fight, I got chatting with some fellow fans in the local pub. While some were intending to pay for the fight, others were going down the Kodi route.

Soon after the conversation switched to IPTV. One of the guys had a subscription and he said that his supplier would be along shortly if anyone wanted a package to watch the fight at home. Of course, I was curious to hear what he had to say since it’s not often this kind of thing is offered ‘offline’.

The guy revealed that he sold more or less exclusively on eBay and called up the page on his phone to show me. The listing made interesting reading.

In common with hundreds of similar IPTV subscription offers easily findable on eBay, the listing offered “All the sports and films you need plus VOD and main UK channels” for the sum of just under £60 per year, which is fairly cheap in the current market. With a non-committal “hmmm” I asked a bit more about the guy’s business and surprisingly he was happy to provide some details.

Like many people offering such packages, the guy was a reseller of someone else’s product. He also insisted that selling access to copyrighted content is OK because it sits in a “gray area”. It’s also easy to keep listings up on eBay, he assured me, as long as a few simple rules are adhered to. Right, this should be interesting.

First of all, sellers shouldn’t be “too obvious” he advised, noting that individual channels or channel lists shouldn’t be listed on the site. Fair enough, but then he said the most important thing of all is to have a disclaimer like his in any listing, written as follows:

“PLEASE NOTE EBAY: THIS IS NOT A DE SCRAMBLER SERVICE, I AM NOT SELLING ANY ILLEGAL CHANNELS OR CHANNEL LISTS NOR DO I REPRESENT ANY MEDIA COMPANY NOR HAVE ACCESS TO ANY OF THEIR CONTENTS. NO TRADEMARK HAS BEEN INFRINGED. DO NOT REMOVE LISTING AS IT IS IN ACCORDANCE WITH EBAY POLICIES.”

Apparently, this paragraph is crucial to keeping listings up on eBay and is the equivalent of kryptonite when it comes to deflecting copyright holders, police, and Trading Standards. Sure enough, a few seconds with Google reveals the same wording on dozens of eBay listings and those offering IPTV subscriptions on external platforms.

It is, of course, absolutely worthless but the IPTV seller insisted otherwise, noting he’d sold “thousands” of subscriptions through eBay without any problems. While a similar logic can be applied to garlic and vampires, a second disclaimer found on many other illicit IPTV subscription listings treads an even more bizarre path.

“THE PRODUCTS OFFERED CAN NOT BE USED TO DESCRAMBLE OR OTHERWISE ENABLE ACCESS TO CABLE OR SATELLITE TELEVISION PROGRAMS THAT BYPASSES PAYMENT TO THE SERVICE PROVIDER. RECEIVING SUBSCRIPTION/BASED TV AIRTIME IS ILLEGAL WITHOUT PAYING FOR IT.”

This disclaimer (which apparently no sellers displaying it have ever read) seems to be have been culled from the Zgemma site, which advertises a receiving device which can technically receive pirate IPTV services but wasn’t designed for the purpose. In that context, the disclaimer makes sense but when applied to dedicated pirate IPTV subscriptions, it’s absolutely ridiculous.

It’s unclear why so many sellers on eBay, Gumtree, Craigslist and other platforms think that these disclaimers are useful. It leads one to the likely conclusion that these aren’t hardcore pirates at all but regular people simply out to make a bit of extra cash who have received bad advice.

What is clear, however, is that selling access to thousands of otherwise subscription channels without permission from copyright owners is definitely illegal in the EU. The European Court of Justice says so (1,2) and it’s been backed up by subsequent cases in the Netherlands.

While the odds of getting criminally prosecuted or sued for reselling such a service are relatively slim, it’s worrying that in 2018 people still believe that doing so is made legal by the inclusion of a paragraph of text. It’s even more worrying that these individuals apparently have no idea of the serious consequences should they become singled out for legal action.

Even more surprisingly, TorrentFreak spoke with a handful of IPTV suppliers higher up the chain who also told us that what they are doing is legal. A couple claimed to be protected by communication intermediary laws, others didn’t want to go into details. Most stopped responding to emails on the topic. Perhaps most tellingly, none wanted to go on the record.

The big take-home here is that following some important EU rulings, knowingly linking to copyrighted content for profit is nearly always illegal in Europe and leaves people open for targeting by copyright holders and the authorities. People really should be aware of that, especially the little guy making a little extra pocket money on eBay.

Of course, people are perfectly entitled to carry on regardless and test the limits of the law when things go wrong. At this point, however, it’s probably worth noting that IPTV provider Ace Hosting recently handed over £600,000 rather than fight the Premier League (1,2) when they clearly had the money to put up a defense.

Given their effectiveness, perhaps they should’ve put up a disclaimer instead?

Source: TF, for the latest info on copyright, file-sharing, torrent sites and more. We also have VPN reviews, discounts, offers and coupons.

Storing Encrypted Credentials In Git

Post Syndicated from Bozho original https://techblog.bozho.net/storing-encrypted-credentials-in-git/

We all know that we should not commit any passwords or keys to the repo with our code (no matter if public or private). Yet, thousands of production passwords can be found on GitHub (and probably thousands more in internal company repositories). Some have tried to fix that by removing the passwords (once they learned it’s not a good idea to store them publicly), but passwords have remained in the git history.

Knowing what not to do is the first and very important step. But how do we store production credentials. Database credentials, system secrets (e.g. for HMACs), access keys for 3rd party services like payment providers or social networks. There doesn’t seem to be an agreed upon solution.

I’ve previously argued with the 12-factor app recommendation to use environment variables – if you have a few that might be okay, but when the number of variables grow (as in any real application), it becomes impractical. And you can set environment variables via a bash script, but you’d have to store it somewhere. And in fact, even separate environment variables should be stored somewhere.

This somewhere could be a local directory (risky), a shared storage, e.g. FTP or S3 bucket with limited access, or a separate git repository. I think I prefer the git repository as it allows versioning (Note: S3 also does, but is provider-specific). So you can store all your environment-specific properties files with all their credentials and environment-specific configurations in a git repo with limited access (only Ops people). And that’s not bad, as long as it’s not the same repo as the source code.

Such a repo would look like this:

project
└─── production
|   |   application.properites
|   |   keystore.jks
└─── staging
|   |   application.properites
|   |   keystore.jks
└─── on-premise-client1
|   |   application.properites
|   |   keystore.jks
└─── on-premise-client2
|   |   application.properites
|   |   keystore.jks

Since many companies are using GitHub or BitBucket for their repositories, storing production credentials on a public provider may still be risky. That’s why it’s a good idea to encrypt the files in the repository. A good way to do it is via git-crypt. It is “transparent” encryption because it supports diff and encryption and decryption on the fly. Once you set it up, you continue working with the repo as if it’s not encrypted. There’s even a fork that works on Windows.

You simply run git-crypt init (after you’ve put the git-crypt binary on your OS Path), which generates a key. Then you specify your .gitattributes, e.g. like that:

secretfile filter=git-crypt diff=git-crypt
*.key filter=git-crypt diff=git-crypt
*.properties filter=git-crypt diff=git-crypt
*.jks filter=git-crypt diff=git-crypt

And you’re done. Well, almost. If this is a fresh repo, everything is good. If it is an existing repo, you’d have to clean up your history which contains the unencrypted files. Following these steps will get you there, with one addition – before calling git commit, you should call git-crypt status -f so that the existing files are actually encrypted.

You’re almost done. We should somehow share and backup the keys. For the sharing part, it’s not a big issue to have a team of 2-3 Ops people share the same key, but you could also use the GPG option of git-crypt (as documented in the README). What’s left is to backup your secret key (that’s generated in the .git/git-crypt directory). You can store it (password-protected) in some other storage, be it a company shared folder, Dropbox/Google Drive, or even your email. Just make sure your computer is not the only place where it’s present and that it’s protected. I don’t think key rotation is necessary, but you can devise some rotation procedure.

git-crypt authors claim to shine when it comes to encrypting just a few files in an otherwise public repo. And recommend looking at git-remote-gcrypt. But as often there are non-sensitive parts of environment-specific configurations, you may not want to encrypt everything. And I think it’s perfectly fine to use git-crypt even in a separate repo scenario. And even though encryption is an okay approach to protect credentials in your source code repo, it’s still not necessarily a good idea to have the environment configurations in the same repo. Especially given that different people/teams manage these credentials. Even in small companies, maybe not all members have production access.

The outstanding questions in this case is – how do you sync the properties with code changes. Sometimes the code adds new properties that should be reflected in the environment configurations. There are two scenarios here – first, properties that could vary across environments, but can have default values (e.g. scheduled job periods), and second, properties that require explicit configuration (e.g. database credentials). The former can have the default values bundled in the code repo and therefore in the release artifact, allowing external files to override them. The latter should be announced to the people who do the deployment so that they can set the proper values.

The whole process of having versioned environment-speific configurations is actually quite simple and logical, even with the encryption added to the picture. And I think it’s a good security practice we should try to follow.

The post Storing Encrypted Credentials In Git appeared first on Bozho's tech blog.

Some quick thoughts on the public discussion regarding facial recognition and Amazon Rekognition this past week

Post Syndicated from Dr. Matt Wood original https://aws.amazon.com/blogs/aws/some-quick-thoughts-on-the-public-discussion-regarding-facial-recognition-and-amazon-rekognition-this-past-week/

We have seen a lot of discussion this past week about the role of Amazon Rekognition in facial recognition, surveillance, and civil liberties, and we wanted to share some thoughts.

Amazon Rekognition is a service we announced in 2016. It makes use of new technologies – such as deep learning – and puts them in the hands of developers in an easy-to-use, low-cost way. Since then, we have seen customers use the image and video analysis capabilities of Amazon Rekognition in ways that materially benefit both society (e.g. preventing human trafficking, inhibiting child exploitation, reuniting missing children with their families, and building educational apps for children), and organizations (enhancing security through multi-factor authentication, finding images more easily, or preventing package theft). Amazon Web Services (AWS) is not the only provider of services like these, and we remain excited about how image and video analysis can be a driver for good in the world, including in the public sector and law enforcement.

There have always been and will always be risks with new technology capabilities. Each organization choosing to employ technology must act responsibly or risk legal penalties and public condemnation. AWS takes its responsibilities seriously. But we believe it is the wrong approach to impose a ban on promising new technologies because they might be used by bad actors for nefarious purposes in the future. The world would be a very different place if we had restricted people from buying computers because it was possible to use that computer to do harm. The same can be said of thousands of technologies upon which we all rely each day. Through responsible use, the benefits have far outweighed the risks.

Customers are off to a great start with Amazon Rekognition; the evidence of the positive impact this new technology can provide is strong (and growing by the week), and we’re excited to continue to support our customers in its responsible use.

-Dr. Matt Wood, general manager of artificial intelligence at AWS

Majority of Canadians Consume Online Content Legally, Survey Finds

Post Syndicated from Andy original https://torrentfreak.com/majority-of-canadians-consume-online-content-legally-survey-finds-180531/

Back in January, a coalition of companies and organizations with ties to the entertainment industries called on local telecoms regulator CRTC to implement a national website blocking regime.

Under the banner of Fairplay Canada, members including Bell, Cineplex, Directors Guild of Canada, Maple Leaf Sports and Entertainment, Movie Theatre Association of Canada, and Rogers Media, spoke of an industry under threat from marauding pirates. But just how serious is this threat?

The results of a new survey commissioned by Innovation Science and Economic Development Canada (ISED) in collaboration with the Department of Canadian Heritage (PCH) aims to shine light on the problem by revealing the online content consumption habits of citizens in the Great White North.

While there are interesting findings for those on both sides of the site-blocking debate, the situation seems somewhat removed from the Armageddon scenario predicted by the entertainment industries.

Carried out among 3,301 Canadians aged 12 years and over, the Kantar TNS study aims to cover copyright infringement in six key content areas – music, movies, TV shows, video games, computer software, and eBooks. Attitudes and behaviors are also touched upon while measuring the effectiveness of Canada’s copyright measures.

General Digital Content Consumption

In its introduction, the report notes that 28 million Canadians used the Internet in the three-month study period to November 27, 2017. Of those, 22 million (80%) consumed digital content. Around 20 million (73%) streamed or accessed content, 16 million (59%) downloaded content, while 8 million (28%) shared content.

Music, TV shows and movies all battled for first place in the consumption ranks, with 48%, 48%, and 46% respectively.

Copyright Infringement

According to the study, the majority of Canadians do things completely by the book. An impressive 74% of media-consuming respondents said that they’d only accessed material from legal sources in the preceding three months.

The remaining 26% admitted to accessing at least one illegal file in the same period. Of those, just 5% said that all of their consumption was from illegal sources, with movies (36%), software (36%), TV shows (34%) and video games (33%) the most likely content to be consumed illegally.

Interestingly, the study found that few demographic factors – such as gender, region, rural and urban, income, employment status and language – play a role in illegal content consumption.

“We found that only age and income varied significantly between consumers who infringed by downloading or streaming/accessing content online illegally and consumers who did not consume infringing content online,” the report reads.

“More specifically, the profile of consumers who downloaded or streamed/accessed infringing content skewed slightly younger and towards individuals with household incomes of $100K+.”

Licensed services much more popular than pirate haunts

It will come as no surprise that Netflix was the most popular service with consumers, with 64% having used it in the past three months. Sites like YouTube and Facebook were a big hit too, visited by 36% and 28% of content consumers respectively.

Overall, 74% of online content consumers use licensed services for content while 42% use social networks. Under a third (31%) use a combination of peer-to-peer (BitTorrent), cyberlocker platforms, or linking sites. Stream-ripping services are used by 9% of content consumers.

“Consumers who reported downloading or streaming/accessing infringing content only are less likely to use licensed services and more likely to use peer-to-peer/cyberlocker/linking sites than other consumers of online content,” the report notes.

Attitudes towards legal consumption & infringing content

In common with similar surveys over the years, the Kantar research looked at the reasons why people consume content from various sources, both legal and otherwise.

Convenience (48%), speed (36%) and quality (34%) were the most-cited reasons for using legal sources. An interesting 33% of respondents said they use legal sites to avoid using illegal sources.

On the illicit front, 54% of those who obtained unauthorized content in the previous three months said they did so due to it being free, with 40% citing convenience and 34% mentioning speed.

Almost six out of ten (58%) said lower costs would encourage them to switch to official sources, with 47% saying they’d move if legal availability was improved.

Canada’s ‘Notice-and-Notice’ warning system

People in Canada who share content on peer-to-peer systems like BitTorrent without permission run the risk of receiving an infringement notice warning them to stop. These are sent by copyright holders via users’ ISPs and the hope is that the shock of receiving a warning will turn consumers back to the straight and narrow.

The study reveals that 10% of online content consumers over the age of 12 have received one of these notices but what kind of effect have they had?

“Respondents reported that receiving such a notice resulted in the following: increased awareness of copyright infringement (38%), taking steps to ensure password protected home networks (27%), a household discussion about copyright infringement (27%), and discontinuing illegal downloading or streaming (24%),” the report notes.

While these are all positives for the entertainment industries, Kantar reports that almost a quarter (24%) of people who receive a notice simply ignore them.

Stream-ripping

Once upon a time, people obtaining music via P2P networks was cited as the music industry’s greatest threat but, with the advent of sites like YouTube, so-called stream-ripping is the latest bogeyman.

According to the study, 11% of Internet users say they’ve used a stream-ripping service. They are most likely to be male (62%) and predominantly 18 to 34 (52%) years of age.

“Among Canadians who have used a service to stream-rip music or entertainment, nearly half (48%) have used stream-ripping sites, one-third have used downloader apps (38%), one-in-seven (14%) have used a stream-ripping plug-in, and one-in-ten (10%) have used stream-ripping software,” the report adds.

Set-Top Boxes and VPNs

Few general piracy studies would be complete in 2018 without touching on set-top devices and Virtual Private Networks and this report doesn’t disappoint.

More than one in five (21%) respondents aged 12+ reported using a VPN, with the main purpose of securing communications and Internet browsing (57%).

A relatively modest 36% said they use a VPN to access free content while 32% said the aim was to access geo-blocked content unavailable in Canada. Just over a quarter (27%) said that accessing content from overseas at a reasonable price was the main motivator.

One in ten (10%) of respondents reported using a set-top box, with 78% stating they use them to access paid-for content. Interestingly, only a small number say they use the devices to infringe.

“A minority use set-top boxes to access other content that is not legal or they are unsure if it is legal (16%), or to access live sports that are not legal or they are unsure if it is legal (11%),” the report notes.

“Individuals who consumed a mix of legal and illegal content online are more likely to use VPN services (42%) or TV set-top boxes (21%) than consumers who only downloaded or streamed/accessed legal content.”

Kantar says that the findings of the report will be used to help policymakers evaluate how Canada’s Copyright Act is coping with a changing market and technological developments.

“This research will provide the necessary information required to further develop copyright policy in Canada, as well as to provide a foundation to assess the effectiveness of the measures to address copyright infringement, should future analysis be undertaken,” it concludes.

The full report can be found here (pdf)

Source: TF, for the latest info on copyright, file-sharing, torrent sites and more. We also have VPN reviews, discounts, offers and coupons.

Hiring a Director of Sales

Post Syndicated from Yev original https://www.backblaze.com/blog/hiring-a-director-of-sales/

Backblaze is hiring a Director of Sales. This is a critical role for Backblaze as we continue to grow the team. We need a strong leader who has experience in scaling a sales team and who has an excellent track record for exceeding goals by selling Software as a Service (SaaS) solutions. In addition, this leader will need to be highly motivated, as well as able to create and develop a highly-motivated, success oriented sales team that has fun and enjoys what they do.

The History of Backblaze from our CEO
In 2007, after a friend’s computer crash caused her some suffering, we realized that with every photo, video, song, and document going digital, everyone would eventually lose all of their information. Five of us quit our jobs to start a company with the goal of making it easy for people to back up their data.

Like many startups, for a while we worked out of a co-founder’s one-bedroom apartment. Unlike most startups, we made an explicit agreement not to raise funding during the first year. We would then touch base every six months and decide whether to raise or not. We wanted to focus on building the company and the product, not on pitching and slide decks. And critically, we wanted to build a culture that understood money comes from customers, not the magical VC giving tree. Over the course of 5 years we built a profitable, multi-million dollar revenue business — and only then did we raise a VC round.

Fast forward 10 years later and our world looks quite different. You’ll have some fantastic assets to work with:

  • A brand millions recognize for openness, ease-of-use, and affordability.
  • A computer backup service that stores over 500 petabytes of data, has recovered over 30 billion files for hundreds of thousands of paying customers — most of whom self-identify as being the people that find and recommend technology products to their friends.
  • Our B2 service that provides the lowest cost cloud storage on the planet at 1/4th the price Amazon, Google or Microsoft charges. While being a newer product on the market, it already has over 100,000 IT and developers signed up as well as an ecosystem building up around it.
  • A growing, profitable and cash-flow positive company.
  • And last, but most definitely not least: a great sales team.

You might be saying, “sounds like you’ve got this under control — why do you need me?” Don’t be misled. We need you. Here’s why:

  • We have a great team, but we are in the process of expanding and we need to develop a structure that will easily scale and provide the most success to drive revenue.
  • We just launched our outbound sales efforts and we need someone to help develop that into a fully successful program that’s building a strong pipeline and closing business.
  • We need someone to work with the marketing department and figure out how to generate more inbound opportunities that the sales team can follow up on and close.
  • We need someone who will work closely in developing the skills of our current sales team and build a path for career growth and advancement.
  • We want someone to manage our Customer Success program.

So that’s a bit about us. What are we looking for in you?

Experience: As a sales leader, you will strategically build and drive the territory’s sales pipeline by assembling and leading a skilled team of sales professionals. This leader should be familiar with generating, developing and closing software subscription (SaaS) opportunities. We are looking for a self-starter who can manage a team and make an immediate impact of selling our Backup and Cloud Storage solutions. In this role, the sales leader will work closely with the VP of Sales, marketing staff, and service staff to develop and implement specific strategic plans to achieve and exceed revenue targets, including new business acquisition as well as build out our customer success program.

Leadership: We have an experienced team who’s brought us to where we are today. You need to have the people and management skills to get them excited about working with you. You need to be a strong leader and compassionate about developing and supporting your team.

Data driven and creative: The data has to show something makes sense before we scale it up. However, without creativity, it’s easy to say “the data shows it’s impossible” or to find a local maximum. Whether it’s deciding how to scale the team, figuring out what our outbound sales efforts should look like or putting a plan in place to develop the team for career growth, we’ve seen a bit of creativity get us places a few extra dollars couldn’t.

Jive with our culture: Strong leaders affect culture and the person we hire for this role may well shape, not only fit into, ours. But to shape the culture you have to be accepted by the organism, which means a certain set of shared values. We default to openness with our team, our customers, and everyone if possible. We love initiative — without arrogance or dictatorship. We work to create a place people enjoy showing up to work. That doesn’t mean ping pong tables and foosball (though we do try to have perks & fun), but it means people are friendly, non-political, working to build a good service but also a good place to work.

Do the work: Ideas and strategy are critical, but good execution makes them happen. We’re looking for someone who can help the team execute both from the perspective of being capable of guiding and organizing, but also someone who is hands-on themselves.

Additional Responsibilities needed for this role:

  • Recruit, coach, mentor, manage and lead a team of sales professionals to achieve yearly sales targets. This includes closing new business and expanding upon existing clientele.
  • Expand the customer success program to provide the best customer experience possible resulting in upsell opportunities and a high retention rate.
  • Develop effective sales strategies and deliver compelling product demonstrations and sales pitches.
  • Acquire and develop the appropriate sales tools to make the team efficient in their daily work flow.
  • Apply a thorough understanding of the marketplace, industry trends, funding developments, and products to all management activities and strategic sales decisions.
  • Ensure that sales department operations function smoothly, with the goal of facilitating sales and/or closings; operational responsibilities include accurate pipeline reporting and sales forecasts.
  • This position will report directly to the VP of Sales and will be staffed in our headquarters in San Mateo, CA.

Requirements:

  • 7 – 10+ years of successful sales leadership experience as measured by sales performance against goals.
    Experience in developing skill sets and providing career growth and opportunities through advancement of team members.
  • Background in selling SaaS technologies with a strong track record of success.
  • Strong presentation and communication skills.
  • Must be able to travel occasionally nationwide.
  • BA/BS degree required

Think you want to join us on this adventure?
Send an email to jobscontact@backblaze.com with the subject “Director of Sales.” (Recruiters and agencies, please don’t email us.) Include a resume and answer these two questions:

  1. How would you approach evaluating the current sales team and what is your process for developing a growth strategy to scale the team?
  2. What are the goals you would set for yourself in the 3 month and 1-year timeframes?

Thank you for taking the time to read this and I hope that this sounds like the opportunity for which you’ve been waiting.

Backblaze is an Equal Opportunity Employer.

The post Hiring a Director of Sales appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

[$] Unprivileged filesystem mounts, 2018 edition

Post Syndicated from corbet original https://lwn.net/Articles/755593/rss

The advent of user namespaces and container technology has made it possible
to extend more root-like powers to unprivileged users in a (we hope) safe
way. One remaining sticking point is the mounting of filesystems, which
has long been fraught with security problems. Work has been proceeding to
allow such mounts for years, and it has gotten a little closer with the
posting of a patch series intended for the 4.18 kernel. But, as an
unrelated discussion has made clear, truly safe unprivileged filesystem
mounting is still a rather distant prospect — at least, if one wants to do
it in the kernel.

Getting Rid of Your Mac? Here’s How to Securely Erase a Hard Drive or SSD

Post Syndicated from Roderick Bauer original https://www.backblaze.com/blog/how-to-wipe-a-mac-hard-drive/

erasing a hard drive and a solid state drive

What do I do with a Mac that still has personal data on it? Do I take out the disk drive and smash it? Do I sweep it with a really strong magnet? Is there a difference in how I handle a hard drive (HDD) versus a solid-state drive (SSD)? Well, taking a sledgehammer or projectile weapon to your old machine is certainly one way to make the data irretrievable, and it can be enormously cathartic as long as you follow appropriate safety and disposal protocols. But there are far less destructive ways to make sure your data is gone for good. Let me introduce you to secure erasing.

Which Type of Drive Do You Have?

Before we start, you need to know whether you have a HDD or a SSD. To find out, or at least to make sure, you click on the Apple menu and select “About this Mac.” Once there, select the “Storage” tab to see which type of drive is in your system.

The first example, below, shows a SATA Disk (HDD) in the system.

SATA HDD

In the next case, we see we have a Solid State SATA Drive (SSD), plus a Mac SuperDrive.

Mac storage dialog showing SSD

The third screen shot shows an SSD, as well. In this case it’s called “Flash Storage.”

Flash Storage

Make Sure You Have a Backup

Before you get started, you’ll want to make sure that any important data on your hard drive has moved somewhere else. OS X’s built-in Time Machine backup software is a good start, especially when paired with Backblaze. You can learn more about using Time Machine in our Mac Backup Guide.

With a local backup copy in hand and secure cloud storage, you know your data is always safe no matter what happens.

Once you’ve verified your data is backed up, roll up your sleeves and get to work. The key is OS X Recovery — a special part of the Mac operating system since OS X 10.7 “Lion.”

How to Wipe a Mac Hard Disk Drive (HDD)

NOTE: If you’re interested in wiping an SSD, see below.

    1. Make sure your Mac is turned off.
    2. Press the power button.
    3. Immediately hold down the command and R keys.
    4. Wait until the Apple logo appears.
    5. Select “Disk Utility” from the OS X Utilities list. Click Continue.
    6. Select the disk you’d like to erase by clicking on it in the sidebar.
    7. Click the Erase button.
    8. Click the Security Options button.
    9. The Security Options window includes a slider that enables you to determine how thoroughly you want to erase your hard drive.

There are four notches to that Security Options slider. “Fastest” is quick but insecure — data could potentially be rebuilt using a file recovery app. Moving that slider to the right introduces progressively more secure erasing. Disk Utility’s most secure level erases the information used to access the files on your disk, then writes zeroes across the disk surface seven times to help remove any trace of what was there. This setting conforms to the DoD 5220.22-M specification.

  1. Once you’ve selected the level of secure erasing you’re comfortable with, click the OK button.
  2. Click the Erase button to begin. Bear in mind that the more secure method you select, the longer it will take. The most secure methods can add hours to the process.

Once it’s done, the Mac’s hard drive will be clean as a whistle and ready for its next adventure: a fresh installation of OS X, being donated to a relative or a local charity, or just sent to an e-waste facility. Of course you can still drill a hole in your disk or smash it with a sledgehammer if it makes you happy, but now you know how to wipe the data from your old computer with much less ruckus.

The above instructions apply to older Macintoshes with HDDs. What do you do if you have an SSD?

Securely Erasing SSDs, and Why Not To

Most new Macs ship with solid state drives (SSDs). Only the iMac and Mac mini ship with regular hard drives anymore, and even those are available in pure SSD variants if you want.

If your Mac comes equipped with an SSD, Apple’s Disk Utility software won’t actually let you zero the hard drive.

Wait, what?

In a tech note posted to Apple’s own online knowledgebase, Apple explains that you don’t need to securely erase your Mac’s SSD:

With an SSD drive, Secure Erase and Erasing Free Space are not available in Disk Utility. These options are not needed for an SSD drive because a standard erase makes it difficult to recover data from an SSD.

In fact, some folks will tell you not to zero out the data on an SSD, since it can cause wear and tear on the memory cells that, over time, can affect its reliability. I don’t think that’s nearly as big an issue as it used to be — SSD reliability and longevity has improved.

If “Standard Erase” doesn’t quite make you feel comfortable that your data can’t be recovered, there are a couple of options.

FileVault Keeps Your Data Safe

One way to make sure that your SSD’s data remains secure is to use FileVault. FileVault is whole-disk encryption for the Mac. With FileVault engaged, you need a password to access the information on your hard drive. Without it, that data is encrypted.

There’s one potential downside of FileVault — if you lose your password or the encryption key, you’re screwed: You’re not getting your data back any time soon. Based on my experience working at a Mac repair shop, losing a FileVault key happens more frequently than it should.

When you first set up a new Mac, you’re given the option of turning FileVault on. If you don’t do it then, you can turn on FileVault at any time by clicking on your Mac’s System Preferences, clicking on Security & Privacy, and clicking on the FileVault tab. Be warned, however, that the initial encryption process can take hours, as will decryption if you ever need to turn FileVault off.

With FileVault turned on, you can restart your Mac into its Recovery System (by restarting the Mac while holding down the command and R keys) and erase the hard drive using Disk Utility, once you’ve unlocked it (by selecting the disk, clicking the File menu, and clicking Unlock). That deletes the FileVault key, which means any data on the drive is useless.

FileVault doesn’t impact the performance of most modern Macs, though I’d suggest only using it if your Mac has an SSD, not a conventional hard disk drive.

Securely Erasing Free Space on Your SSD

If you don’t want to take Apple’s word for it, if you’re not using FileVault, or if you just want to, there is a way to securely erase free space on your SSD. It’s a little more involved but it works.

Before we get into the nitty-gritty, let me state for the record that this really isn’t necessary to do, which is why Apple’s made it so hard to do. But if you’re set on it, you’ll need to use Apple’s Terminal app. Terminal provides you with command line interface access to the OS X operating system. Terminal lives in the Utilities folder, but you can access Terminal from the Mac’s Recovery System, as well. Once your Mac has booted into the Recovery partition, click the Utilities menu and select Terminal to launch it.

From a Terminal command line, type:

diskutil secureErase freespace VALUE /Volumes/DRIVE

That tells your Mac to securely erase the free space on your SSD. You’ll need to change VALUE to a number between 0 and 4. 0 is a single-pass run of zeroes; 1 is a single-pass run of random numbers; 2 is a 7-pass erase; 3 is a 35-pass erase; and 4 is a 3-pass erase. DRIVE should be changed to the name of your hard drive. To run a 7-pass erase of your SSD drive in “JohnB-Macbook”, you would enter the following:

diskutil secureErase freespace 2 /Volumes/JohnB-Macbook

And remember, if you used a space in the name of your Mac’s hard drive, you need to insert a leading backslash before the space. For example, to run a 35-pass erase on a hard drive called “Macintosh HD” you enter the following:

diskutil secureErase freespace 3 /Volumes/Macintosh\ HD

Something to remember is that the more extensive the erase procedure, the longer it will take.

When Erasing is Not Enough — How to Destroy a Drive

If you absolutely, positively need to be sure that all the data on a drive is irretrievable, see this Scientific American article (with contributions by Gleb Budman, Backblaze CEO), How to Destroy a Hard Drive — Permanently.

The post Getting Rid of Your Mac? Here’s How to Securely Erase a Hard Drive or SSD appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

Pirate IPTV Sellers Sign Abstention Agreements Under Pressure From BREIN

Post Syndicated from Andy original https://torrentfreak.com/pirate-iptv-sellers-sign-abstention-agreement-under-pressure-from-brein-180528/

Earlier this month, Dutch anti-piracy outfit BREIN revealed details of its case against Netherlands-based company Leaper Beheer BV.

BREIN’s complaint, which was filed at the Limburg District Court in Maastricht, claimed that
Leaper sold access to unlicensed live TV streams and on-demand movies. Around 4,000 live channels and 1,000 movies were included in the package, which was distributed to customers in the form of an .M3U playlist.

BREIN said that distribution of the playlist amounted to a communication to the public in contravention of the EU Copyright Directive. In its defense, Leaper argued that it is not a distributor of content itself and did not make anything available that wasn’t already public.

In a detailed ruling the Court sided with BREIN, noting that Leaper communicated works to a new audience that wasn’t taken into account when the content’s owners initially gave permission for their work to be distributed to the public.

The Court ordered Leaper to stop providing access to the unlicensed streams or face penalties of 5,000 euros per IPTV subscription sold, link offered, or days exceeded, to a maximum of one million euros. Further financial penalties were threatened for non-compliance with other aspects of the ruling.

In a fresh announcement Friday, BREIN revealed that three companies and their directors (Leaper included) have signed agreements to cease-and-desist, in order to avert summary proceedings. According to BREIN, the companies are the biggest sellers of pirate IPTV subscriptions in the Netherlands.

In addition to Leaper Beheer BV, Growler BV, DITisTV and their respective directors are bound by a number of conditions in their agreements but primarily to cease-and-desist offering hyperlinks or other technical means to access protected works belonging to BREIN’s affiliates and their members.

Failure to comply with the terms of the agreement will see the companies face penalties of 10,000 euros per infringement or per day (or part thereof).

DITisTV’s former website now appears to sell shoes and a search for the company using Google doesn’t reveal many flattering results. Consumer website Consumentenbond.nl enjoys the top spot with an article reporting that it received 300 complaints about DITisTV.

“The complainants report that after they have paid, they have not received their order, or that they were not given a refund if they sent back a malfunctioning media player. Some consumers have been waiting for their money for several months,” the article reads.

According to the report, DiTisTV pulled the plug on its website last June, probably in response to the European Court of Justice ruling which found that selling piracy-configured media players is illegal.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and more. We also have VPN reviews, discounts, offers and coupons.

The FBI tells everybody to reboot their router

Post Syndicated from corbet original https://lwn.net/Articles/755741/rss

This CERT
advisory
warns of over 500,000 home routers that have been compromised
by the VPNFilter malware and is advising everybody to reboot their routers
to (partially) remove it. This Talos
Intelligence page
has a lot more information about VPNFilter, though a
lot apparently remains unknown. “At the time of this publication, we
do not have definitive proof on how the threat actor is exploiting the
affected devices. However, all of the affected makes/models that we have
uncovered had well-known, public vulnerabilities. Since advanced threat
actors tend to only use the minimum resources necessary to accomplish their
goals, we assess with high confidence that VPNFilter required no zero-day
exploitation techniques.

Putin Asked to Investigate Damage Caused By Telegram Web-Blocking

Post Syndicated from Andy original https://torrentfreak.com/putin-asked-to-investigate-damage-caused-by-telegram-web-blocking-180526/

After a Moscow court gave the go-ahead for Telegram to be banned in Russia last month, the Internet became a battleground.

On the instructions of telecoms watchdog Roscomnadzor, ISPs across Russia tried to block Telegram by blackholing millions of IP addresses. The effect was both dramatic and pathetic. While Telegram remained stubbornly online, countless completely innocent services suffered outages as Roscomnadzor charged ahead with its mission.

Over the past several weeks, Roscomnadzor has gone some way to clean up the mess, partly by removing innocent Google and Amazon IP addresses from Russia’s blacklist. However, the collateral damage was so widespread it’s called into question the watchdog’s entire approach to web-blockades and whether they should be carried out at any cost.

This week, thanks to an annual report presented to President Vladimir Putin by business ombudsman Boris Titov, the matter looks set to be escalated. ‘The Book of Complaints and Suggestions of Russian Business’ contains comments from Internet ombudsman Dmitry Marinichev, who says that the Prosecutor General’s Office should launch an investigation into Roscomnadzor’s actions.

Marinichev said that when attempting to take down Telegram using aggressive technical means, Roscomnadzor relied upon “its own interpretation of court decisions” to provide guidance, TASS reports.

“When carrying out blockades of information resources, Roskomnadzor did not assess the related damage caused to them,” he said.

More than 15 million IP addresses were blocked, many of them with functions completely unrelated to the operations of Telegram. Marinichev said that the consequences were very real for those who suffered collateral damage.

“[The blocking led] to a temporary inaccessibility of Internet resources of a number of Russian enterprises in the Internet sector, including several banks and government information resources,” he reported.

In advice to the President, Marinichev suggests that the Prosecutor General’s Office should look into “the legality and validity of Roskomnadzor’s actions” which led to the “violation of availability of information resources of commercial companies” and “threatened the integrity, sustainability, and functioning of the unified telecommunications network of the Russian Federation and its critical information infrastructure.”

Early May, it was reported that in addition to various web services, around 50 VPN, proxy and anonymization platforms had been blocked for providing access to Telegram. In a May 22 report, that number had swelled to more than 80 although 10 were later unblocked after they stopped providing access to the messaging platform.

This week, Roscomnadzor has continued with efforts to block access to torrent and streaming platforms. In a new wave of action, the telecoms watchdog ordered ISPs to block at least 47 mirrors and proxies providing access to previously blocked sites.

Source: TF, for the latest info on copyright, file-sharing, torrent sites and more. We also have VPN reviews, discounts, offers and coupons.

Welcome Jack — Data Center Tech

Post Syndicated from Yev original https://www.backblaze.com/blog/welcome-jack-data-center-tech/

As we shoot way past 500 petabytes of data stored, we need a lot of helping hands in the data center to keep those hard drives spinning! We’ve been hiring quite a lot, and our latest addition is Jack. Lets learn a bit more about him, shall we?

What is your Backblaze Title?
Data Center Tech

Where are you originally from?
Walnut Creek, CA until 7th grade when the family moved to Durango, Colorado.

What attracted you to Backblaze?
I had heard about how cool the Backblaze community is and have always been fascinated by technology.

What do you expect to learn while being at Backblaze?
I expect to learn a lot about how our data centers run and all of the hardware behind it.

Where else have you worked?
Garrhs HVAC as an HVAC Installer and then Durango Electrical as a Low Volt Technician.

Where did you go to school?
Durango High School and then Montana State University.

What’s your dream job?
I would love to be a driver for the Audi Sport. Race cars are so much fun!

Favorite place you’ve traveled?
Iceland has definitely been my favorite so far.

Favorite hobby?
Video games.

Of what achievement are you most proud?
Getting my Eagle Scout badge was a tough, but rewarding experience that I will always cherish.

Star Trek or Star Wars?
Star Wars.

Coke or Pepsi?
Coke…I know, it’s bad.

Favorite food?
Thai food.

Why do you like certain things?
I tend to warm up to things the more time I spend around them, although I never really know until it happens.

Anything else you’d like to tell us?
I’m a friendly car guy who will always be in love with my European cars and I really enjoy the Backblaze community!

We’re happy you joined us Out West! Welcome aboard Jack!

The post Welcome Jack — Data Center Tech appeared first on Backblaze Blog | Cloud Storage & Cloud Backup.

Enchanting images with Inky Lines, a Pi‑powered polargraph

Post Syndicated from Helen Lynn original https://www.raspberrypi.org/blog/enchanting-images-inky-lines-pi-powered-polargraph/

A hanging plotter, also known as a polar plotter or polargraph, is a machine for drawing images on a vertical surface. It does so by using motors to control the length of two cords that form a V shape, supporting a pen where they meet. We’ve featured one on this blog before: Norbert “HomoFaciens” Heinz’s video is a wonderfully clear introduction to how a polargraph works and what you have to consider when you’re putting one together.

Today, we look at Inky Lines, by John Proudlock. With it, John is creating a series of captivating and beautiful pieces, and with his most recent work, each rendering of an image is unique.

The Inky Lines plotter draws a flock of seagulls in blue ink on white paper. The print head is suspended near the bottom left corner of the image, as the pen inks the wing of a gull

An evolving project

The project isn’t new – John has been working on it for at least a couple of years – but it is constantly evolving. When we first spotted it, John had just implemented code to allow the plotter to produce mesmeric, spiralling patterns.

A blue spiral pattern featuring overlapping "bubbles"
A dense pink spiral pattern, featuring concentric circles and reminiscent of a mandala
A blue spirograph-type pattern formed of large overlapping squares, each offset from its neighbour by a few degrees, producing a four-spiral-armed "galaxy" shape where lines overlap. The plotter's print head is visible in a corner of the image

But we’re skipping ahead. Let’s go back to the beginning.

From pixels to motor movements

John starts by providing an image, usually no more than 100 pixels wide, to a Raspberry Pi. Custom software that he wrote evaluates the darkness of each pixel and selects a pattern of a suitable density to represent it.

The two cords supporting the plotter’s pen are wound around the shafts of two stepper motors, such that the movement of the motors controls the length of the cords: the program next calculates how much each motor must move in order to produce the pattern. The Raspberry Pi passes corresponding instructions to two motor circuits, which transform the signals to a higher voltage and pass them to the stepper motors. These turn by very precise amounts, winding or unwinding the cords and, very slowly, dragging the pen across the paper.

A Raspberry Pi in a case, with a wide flex connected to a GPIO header
The Inky Lines plotter's print head, featuring cardboard and tape, draws an apparently random squiggle
A large area of apparently random pattern drawn by the plotter

John explains,

Suspended in-between the two motors is a print head, made out of a new 3-d modelling material I’ve been prototyping called cardboard. An old coat hanger and some velcro were also used.

(He’s our kind of maker.)

Unique images

The earlier drawings that John made used a repeatable method to render image files as lines on paper. That is, if the machine drew the same image a number of times, each copy would be identical. More recently, though, he has been using a method that yields random movements of the pen:

The pen point is guided around the image, but moves to each new point entirely at random. Up close this looks like a chaotic squiggle, but from a distance of a couple of meters, the human eye (and brain) make order from the chaos and view an infinite number of shades and a smoother, less mechanical image.

An apparently chaotic squiggle

This method means that no matter how many times the polargraph repeats the same image, each copy will be unique.

A gallery of work

Inky Lines’ website and its Instagram feed offer a collection of wonderful pieces John has drawn with his polargraph, and he discusses the different techniques and types of image that he is exploring.

A 3 x 3 grid of varied and colourful images from inkylinespolargraph's Instagram feed

They range from holiday photographs, processed to extract particular features and rendered in silhouette, to portraits, made with a single continuous line that can be several hundred metres long, to generative images spirograph images like those pictured above, created by an algorithm rather than rendered from a source image.

The post Enchanting images with Inky Lines, a Pi‑powered polargraph appeared first on Raspberry Pi.