Tag Archives: doom

Weekly roundup: Pixels and art

Post Syndicated from Eevee original https://eev.ee/dev/2017/05/15/weekly-roundup-pixels-and-art/

  • fox flux: I spent a preposterous amount of time spriting, and somehow came out the other end with like… five more sprites. I’m having a tough time designing abstract tiles, too. This is going to take forever.

  • game boy hackery: Pokémon Red and Blue didn’t have unique small sprites for every Pokémon in your party list; instead there were half a dozen general shapes like “generic insect” and “generic snake”.

    A full custom set of icons in that style, by Emi Monserrate, floated past my radar a week ago.

    It seemed like an obvious and interesting challenge to try putting those sprites in the games. So I did.

  • art: I doodled, just, a whole bunch of stuff. Some of it was okay! I oughta produce some actual finished work sometime, for real.

  • blog: I got rid of the Disqus ads. Sorry about that.

  • potluck: You may recall that I’m doing TILESET POTLUCK, a project where people contribute tiles (without being able to see what’s been submitted so far) and I make a game out of whatever I get. This week I got a jump on the actual game by spending a day making a tiny baby environment with Phaser, which seems pretty neat so far.

  • flora: I wrote a third of a short story as a way to get myself jumpstarted on writing, since I still have a bunch of it to do this month. Then I went to bed, so, not sure how well that worked.

  • stream: I streamed the middle third of Doom II The Way id Did. Very exciting.

Still have infinite art to do, but I’m starting to get somewhere. Really need to get started on some writing, so I’m gonna go do that now.

Weekly roundup: HP restored

Post Syndicated from Eevee original https://eev.ee/dev/2017/04/18/weekly-roundup-hp-restored/

I spent a good chunk of the week sick and/or becoming nocturnal, so not a lot of actual work resulted. I’m feeling better now, though, and getting back into the swing of things.

  • art: Did a lot of doodling, since that’s fairly low-energy. Probably the best thing to come out of it was some character design slash concept art for Isaac’s Descent HD.

  • games: Streamed some more Eternal Doom. Seems like every third map has some incredibly obtuse secret required for progression.

  • book: Once I was over the plague, I got back into this, woo. Wasted too much time squabbling with LaTeX before conceding defeat. Working on explaining a bad approach to collision detection at the moment.

Now I can finally get to my plans for April: working on the book more and exploring art/music.

Weekly roundup: Fever dreams

Post Syndicated from Eevee original https://eev.ee/dev/2017/04/11/weekly-roundup-fever-dreams/

I seem to be sick! Again? All of my cycles are devoted to antivirus. Please forgive my brevity.

  • art: Did some final toodling around in Blender and made myself a really bad ironic 90s avatar out of my Eevee model. Speedpainted a rainy view out my window. Doodled or whatever.

  • games: Streamed Eternal Doom and some other stuff a few times, for probably far too long, but whatever it’s fun. Played through Axiom Verge, which has gotten some gears a-turning.

  • blog: Spent probably more than half the week working on a post for Sitepoint, plus a bit of work on another about fox flux, neither of which are published yet.

Weekly roundup: Slow catchup

Post Syndicated from Eevee original https://eev.ee/dev/2017/03/12/weekly-roundup-slow-catchup/

Trying to catch up on stuff and also relax a bit, simultaneously, and the results are confusing.

  • blog: I wrote a late sponsored post on utopia. Also some work on a couple others I have in mind.

  • idchoppers: On a total whim, I started writing a WAD parsing and whatnot library in Rust, called idchoppers. It doesn’t do a whole lot yet and it’s kind of a mess while I figure out how to design types in Rust, but it’s enough that I managed to flip every map in Doom 2 and dump MAP01 to SVG.

  • games: I played a lot of Strawberry Jam games, and I think I’m still only halfway through, yikes.

I feel awake and decently-rested, but somehow still drained. Oof.

Building Smarter Ransomware

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2017/03/building_smarte.html

Matthew Green and students speculate on what truly well-designed ransomware system could look like:

Most modern ransomware employs a cryptocurrency like Bitcoin to enable the payments that make the ransom possible. This is perhaps not the strongest argument for systems like Bitcoin — and yet it seems unlikely that Bitcoin is going away anytime soon. If we can’t solve the problem of Bitcoin, maybe it’s possible to use Bitcoin to make “more reliable” ransomware.

[…]

Recall that in the final step of the ransom process, the ransomware operator must deliver a decryption key to the victim. This step is the most fraught for operators, since it requires them to manage keys and respond to queries on the Internet. Wouldn’t it be better for operators if they could eliminate this step altogether?

[…]

At least in theory it might be possible to develop a DAO that’s funded entirely by ransomware payments — and in turn mindlessly contracts real human beings to develop better ransomware, deploy it against human targets, and…rinse repeat. It’s unlikely that such a system would be stable in the long run ­ humans are clever and good at destroying dumb things ­ but it might get a good run.

One of the reasons society hasn’t destroyed itself is that people with intelligence and skills tend to not be criminals for a living. If it ever became a viable career path, we’re doomed.

In defense of the doomsday-minded super-rich

Post Syndicated from Michal Zalewski original http://lcamtuf.blogspot.com/2017/02/in-defense-of-doomsday-minded-super-rich.html

Several days ago, The New Yorker ran a lengthy story titled
“Doomsday prep for the super-rich”. The article revealed that some of the Silicon Valley’s most successful entrepreneurs – including the execs from Yahoo, Facebook, and Reddit – are planning ahead for extreme emergencies, “Doomsday Preppers” style.

The article made quite a few waves in the tech world – and invited near-universal ridicule and scorn. People sneered at the comical excess of blast-proof bunkers and bug-out helicopters, as if a cataclysm that kills off most of us would somehow spare the nouveaux riches. Hardcore survivalists gleefully proclaimed that the highly-paid armed guards would turn on their employers to save own families. Conservatives rolled their eyes at the story of a VC who is preparing for the collapse of the civilized world but finds guns a bit too icky for his taste. Progressives were repulsed by the immorality of the world’s wealthiest people wanting to hide when the angry underclass finally takes it to the streets. In short, no matter where you stood, the story had something for you to hate.

My first instinct was to join the fray; if nothing else, it’s cathartic to have fun at the expense of people who are far wealthier and far more powerful than we could ever be. Sure, I have written about the merits of common-sense emergency preparedness, but to me, it meant having a rainy day fund and a fire extinguisher, not holing up in a decommissioned ICBM silo with 10,000 rounds of ammo and a pallet of canned cheese. Now hold my beer and let me throw the first stone!

But then, I realized that the article in The New Yorker is a human interest piece; it is meant to entertain us and has no other reason to exist. The author is trying to show us a dazzling world that is out of ordinary and out of reach. It may be that the profiled execs spend most of their time planning ahead for far more pedestrian risks, but no sane newspaper would publish a multi-page expose about the brand of fire extinguishers or tarp favored by the ultra-rich. The readers want to read about helicopters and ICBM silos instead – and so the author obliges.

It is also a fallacy to look at the cost of purchases outlined in the article in absolute terms. For us, spending $5M on a luxury compound and a helicopter may seem insane – but for a person with hundreds of millions in the bank, such an investment would be just 1% of their wealth – a reasonable price to pay for insurance against unlikely but somewhat plausible risks. In terms of relative financial impact, it is no different than a person with $10k in the bank spending $100 on a fire extinguisher and some energy bars – hardly a controversial thing to do.

What’s more, although we’re living in a period of unprecedented prosperity and calm, there’s no denying that in the history of mankind, revolutions happen with near-clockwork regularity. We had quite a few in the past one hundred years alone – and when the time comes, it’s usually the heads of the variously-defined aristocracy that roll. Angry mobs are unlikely to torch down Joe Prepper’s cookie-cutter suburban neighborhood, but being near the top of the social ladder carries some distinct risk. We can have a debate about the complicity of the elites, or the comparative significance of this risk versus the plight of other social classes – but either way, the paranoia of the rich may be more grounded in reality than it seems.

Of course, an argument can be made that preparing for the collapse of the society is immoral when their wealth could be better spent on trying to bridge the income gap or otherwise make the world a more harmonious place. It is an interesting claim, but it rings a bit hollow to me. We would not deny the rich the right to buy a fire extinguisher or a bug-out bicycle; our outrage is rather conveniently reserved for the purchases we can’t afford. But more importantly, prepping and philanthropy are not mutually exclusive; in fact, I suspect that some of the folks mentioned in the article spend far more on trying to help the less fortunate than they are spending on canned cheese. Whether this can make a difference, and whether they should be doing more, is a different story.

Eevee gained 2611 experience points

Post Syndicated from Eevee original https://eev.ee/blog/2017/01/21/eevee-gained-2611-experience-points/

Eevee grew to level 30!

Wow! What an incredible fucking year. I’ve never done so much in my life.

I made a video game! Twice! Thrice, even, if you count the one I just finished, which was done on the week surrounding my birthday. Mel and I made Under Construction on the PICO-8 in a few weeks, then I used its engine to make Isaac’s Descent on my own for Ludum Dare, and we just published the more substantial LÖVE-based NEON PHASE for my own week-long game jam. I’ve also got a couple other things in progress, and Mel is at this very second working on NEON PHASE 2.

And we made these from scratch. I wrote a little collision engine! It even mostly works! That’s amazing and I’m so happy about it. It’s also great that Mel and I have something we can actually work on together; before, the most collaborating we really did was on web design.

My interest in Doom waxes and wanes, but I did find time to finally make a real map and later make some speedmaps (which I realize I never fully published, oops!). I even put some effort into embedding Lua in ZDoom, which is an interesting side project but probably not something I’m going to actively champion.

I wrote a few Twitter bots! @perlin_noise puts my post on Perlin noise to use by tweeting images, GIFs, and emoji grids generated from various kinds of Perlin noise. @unicodeveryword tweets every word in the English language that’s also in my wordlist, slightly scrambled by Unicode shenanigans. @calloutbot is mostly an injoke.

I wrote 45 posts comprising 195k words, which is about four novels, and that’s not counting the couple of articles I was paid to write a few months ago. I have Patreon to thank for getting me to write more often, though I recently dropped the writing obligation so I can spend more time on games and a book and other stuff.

Oh, yes, I actually started writing a book, in the sense that I have (quite a bit) more than a few pages written and a concept that I like and am sticking to.

I dipped a toe into music, however briefly, and managed to produce a simple song that’s not too bad. Promising!

I feel like I’ve been slacking a lot on art lately, but looking back, this year saw my first animation (and several more after that), a mountain of ink-on-paper work for Inktober, and of course a lot of exploration of pixel art for games. I haven’t done as much digital illustrating as I’d like in the last few months, perhaps, but I’m definitely creating stuff I’d never have imagined I could do two years ago.

Veekun? Ah, hm. I did write some gen 1 dumping stuff, much of which can be reused for gen 2, so that’s good. Some of the code got modernized a bit, and I made some good progress on gen 6/7. I also wrote that model viewer, which is pretty dang cool. So, still making progress, just slowly. It’s obviously not something that holds my attention as strongly nowadays, but I’d still like to get the site redone — hopefully that’ll make it easier to understand and work on, both for me and other people.

Even after all this, I’m surely forgetting a few more minor things! I’m really happy about how far I’ve come in a year, and I can’t wait to see what this coming year has in store.

Eevee gained 2611 experience points

Post Syndicated from Eevee original https://eev.ee/blog/2017/01/21/eevee-gained-2611-experience-points/

Eevee grew to level 30!

Wow! What an incredible fucking year. I’ve never done so much in my life.

I made a video game! Twice! Thrice, even, if you count the one I just finished, which was done on the week surrounding my birthday. Mel and I made Under Construction on the PICO-8 in a few weeks, then I used its engine to make Isaac’s Descent on my own for Ludum Dare, and we just published the more substantial LÖVE-based NEON PHASE for my own week-long game jam. I’ve also got a couple other things in progress, and Mel is at this very second working on NEON PHASE 2.

And we made these from scratch. I wrote a little collision engine! It even mostly works! That’s amazing and I’m so happy about it. It’s also great that Mel and I have something we can actually work on together; before, the most collaborating we really did was on web design.

My interest in Doom waxes and wanes, but I did find time to finally make a real map and later make some speedmaps (which I realize I never fully published, oops!). I even put some effort into embedding Lua in ZDoom, which is an interesting side project but probably not something I’m going to actively champion.

I wrote a few Twitter bots! @perlin_noise puts my post on Perlin noise to use by tweeting images, GIFs, and emoji grids generated from various kinds of Perlin noise. @unicodeveryword tweets every word in the English language that’s also in my wordlist, slightly scrambled by Unicode shenanigans. @calloutbot is mostly an injoke.

I wrote 45 posts comprising 195k words, which is about four novels, and that’s not counting the couple of articles I was paid to write a few months ago. I have Patreon to thank for getting me to write more often, though I recently dropped the writing obligation so I can spend more time on games and a book and other stuff.

Oh, yes, I actually started writing a book, in the sense that I have (quite a bit) more than a few pages written and a concept that I like and am sticking to.

I dipped a toe into music, however briefly, and managed to produce a simple song that’s not too bad. Promising!

I feel like I’ve been slacking a lot on art lately, but looking back, this year saw my first animation (and several more after that), a mountain of ink-on-paper work for Inktober, and of course a lot of exploration of pixel art for games. I haven’t done as much digital illustrating as I’d like in the last few months, perhaps, but I’m definitely creating stuff I’d never have imagined I could do two years ago.

Veekun? Ah, hm. I did write some gen 1 dumping stuff, much of which can be reused for gen 2, so that’s good. Some of the code got modernized a bit, and I made some good progress on gen 6/7. I also wrote that model viewer, which is pretty dang cool. So, still making progress, just slowly. It’s obviously not something that holds my attention as strongly nowadays, but I’d still like to get the site redone — hopefully that’ll make it easier to understand and work on, both for me and other people.

Even after all this, I’m surely forgetting a few more minor things! I’m really happy about how far I’ve come in a year, and I can’t wait to see what this coming year has in store.

Are We Becoming More Moral Faster Than We’re Becoming More Dangerous?

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2017/01/are_we_becoming.html

In The Better Angels of Our Nature, Steven Pinker convincingly makes the point that by pretty much every measure you can think of, violence has declined on our planet over the long term. More generally, “the world continues to improve in just about every way.” He’s right, but there are two important caveats.

One, he is talking about the long term. The trend lines are uniformly positive across the centuries and mostly positive across the decades, but go up and down year to year. While this is an important development for our species, most of us care about changes year to year — and we can’t make any predictions about whether this year will be better or worse than last year in any individual measurement.

The second caveat is both more subtle and more important. In 2013, I wrote about how technology empowers attackers. By this measure, the world is getting more dangerous:

Because the damage attackers can cause becomes greater as technology becomes more powerful. Guns become more harmful, explosions become bigger, malware becomes more pernicious… and so on. A single attacker, or small group of attackers, can cause more destruction than ever before.

This is exactly why the whole post-9/11 weapons-of-mass-destruction debate was so overwrought: Terrorists are scary, terrorists flying airplanes into buildings are even scarier, and the thought of a terrorist with a nuclear bomb is absolutely terrifying.

Pinker’s trends are based both on increased societal morality and better technology, and both are based on averages: the average person with the average technology. My increased attack capability trend is based on those two trends as well, but on the outliers: the most extreme person with the most extreme technology. Pinker’s trends are noisy, but over the long term they’re strongly linear. Mine seem to be exponential.

When Pinker expresses optimism that the overall trends he identifies will continue into the future, he’s making a bet. He’s betting that his trend lines and my trend lines won’t cross. That is, that our society’s gradual improvement in overall morality will continue to outpace the potentially exponentially increasing ability of the extreme few to destroy everything. I am less optimistic:

But the problem isn’t that these security measures won’t work — even as they shred our freedoms and liberties — it’s that no security is perfect.

Because sooner or later, the technology will exist for a hobbyist to explode a nuclear weapon, print a lethal virus from a bio-printer, or turn our electronic infrastructure into a vehicle for large-scale murder. We’ll have the technology eventually to annihilate ourselves in great numbers, and sometime after, that technology will become cheap enough to be easy.

As it gets easier for one member of a group to destroy the entire group, and the group size gets larger, the odds of someone in the group doing it approaches certainty. Our global interconnectedness means that our group size encompasses everyone on the planet, and since government hasn’t kept up, we have to worry about the weakest-controlled member of the weakest-controlled country. Is this a fundamental limitation of technological advancement, one that could end civilization? First our fears grip us so strongly that, thinking about the short term, we willingly embrace a police state in a desperate attempt to keep us safe; then, someone goes off and destroys us anyway?

Clearly we’re not at the point yet where any of these disaster scenarios have come to pass, and Pinker rightly expresses skepticism when he says that historical doomsday scenarios have so far never come to pass. But that’s the thing about exponential curves; it’s hard to predict the future from the past. So either I have discovered a fundamental problem with any intelligent individualistic species and have therefore explained the Fermi Paradox, or there is some other factor in play that will ensure that the two trend lines won’t cross.

Weekly roundup: Freedom

Post Syndicated from Eevee original https://eev.ee/dev/2016/12/06/weekly-roundup-freedom/

  • zdoom: On a total whim, I resurrected half of an old branch that puts sloped 3D floors in the software renderer. It kinda draws them, but with no textures.

  • blog: I wrote a thing about not copying C which was surprisingly popular.

  • sylph: I accidentally spent 45 minutes writing a microscopic parser for a language that can only print string literals.

  • patreon: I finished up some revamping of my Patreon — the wall of text is now a short and straightforward stack of images, and I dropped the blogging milestones. I’m no longer obliged to write X posts per month, huzzah.

  • art: I pixel-drew some new veekun version icons, which may or may not go live. Also drew a December avatar.

  • veekun: I got started on adapting my ORAS dumping code for Sun and Moon, and have box and dex sprites 90% dumped. Text is mostly done as well.

  • art: I did a few pixels, which you may or not be seeing in the near future.

veekun effort continues (as I scramble to actually finish the game so I don’t spoil myself). Also trying to, uh, remember how to draw?

Weekly roundup: Freedom

Post Syndicated from Eevee original https://eev.ee/dev/2016/12/06/weekly-roundup-freedom/

  • zdoom: On a total whim, I resurrected half of an old branch that puts sloped 3D floors in the software renderer. It kinda draws them, but with no textures.

  • blog: I wrote a thing about not copying C which was surprisingly popular.

  • sylph: I accidentally spent 45 minutes writing a microscopic parser for a language that can only print string literals.

  • patreon: I finished up some revamping of my Patreon — the wall of text is now a short and straightforward stack of images, and I dropped the blogging milestones. I’m no longer obliged to write X posts per month, huzzah.

  • art: I pixel-drew some new veekun version icons, which may or may not go live. Also drew a December avatar.

  • veekun: I got started on adapting my ORAS dumping code for Sun and Moon, and have box and dex sprites 90% dumped. Text is mostly done as well.

  • art: I did a few pixels, which you may or not be seeing in the near future.

veekun effort continues (as I scramble to actually finish the game so I don’t spoil myself). Also trying to, uh, remember how to draw?

Weekly roundup: Arguing on the internet

Post Syndicated from Eevee original https://eev.ee/dev/2016/11/28/weekly-roundup-arguing-on-the-internet/

November is improving.

  • zdoom: On a whim I went back to playing with my experiment of embedding Lua in ZDoom. I tracked down a couple extremely subtle and frustrating bugs with the Lua binding library I was using, and I made it possible to define new actor classes from Lua and delay a script executed from a switch. Those were two huge requirements, so that’s pretty okay progress.

  • blog: Zed Shaw said some appalling nonsense and I felt compelled to correct it. I also wrote about some of the interesting issues with that whole Lua-in-ZDoom thing. I have a final post half-done as well.

  • art: I did a few pixels, which you may or not be seeing in the near future.

Mostly writing and Pokémon, then. But I’m clearing my plate behind the scenes. Stuff’s a-brewin’.

Weekly roundup: Arguing on the internet

Post Syndicated from Eevee original https://eev.ee/dev/2016/11/28/weekly-roundup-arguing-on-the-internet/

November is improving.

  • zdoom: On a whim I went back to playing with my experiment of embedding Lua in ZDoom. I tracked down a couple extremely subtle and frustrating bugs with the Lua binding library I was using, and I made it possible to define new actor classes from Lua and delay a script executed from a switch. Those were two huge requirements, so that’s pretty okay progress.

  • blog: Zed Shaw said some appalling nonsense and I felt compelled to correct it. I also wrote about some of the interesting issues with that whole Lua-in-ZDoom thing. I have a final post half-done as well.

  • art: I did a few pixels, which you may or not be seeing in the near future.

Mostly writing and Pokémon, then. But I’m clearing my plate behind the scenes. Stuff’s a-brewin’.

Embedding Lua in ZDoom

Post Syndicated from Eevee original https://eev.ee/blog/2016/11/26/embedding-lua-in-zdoom/

I’ve spent a little time trying to embed a Lua interpreter in ZDoom. I didn’t get too far yet; it’s just an experimental thing I poke at every once and a while. The existing pile of constraints makes it an interesting problem, though.

Background

ZDoom is a “source port” (read: fork) of the Doom engine, with all the changes from the commercial forks merged in (mostly Heretic, Hexen, Strife), and a lot of internal twiddles exposed. It has a variety of mechanisms for customizing game behavior; two are major standouts.

One is ACS, a vaguely C-ish language inherited from Hexen. It’s mostly used to automate level behavior — at the simplest, by having a single switch perform multiple actions. It supports the usual loops and conditionals, it can store data persistently, and ZDoom exposes a number of functions to it for inspecting and altering the state of the world, so it can do some neat tricks. Here’s an arbitrary script from my DUMP2 map.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
script "open_church_door" (int tag)
{
    // Open the door more quickly on easier skill levels, so running from the
    // arch-vile is a more viable option
    int skill = GameSkill();
    int speed;
    if (skill < SKILL_NORMAL)
        speed = 64;  // blazing door speed
    else if (skill == SKILL_NORMAL)
        speed = 16;  // normal door speed
    else
        speed = 8;  // very dramatic door speed

    Door_Raise(tag, speed, 68);  // double usual delay
}

However, ZDoom doesn’t actually understand the language itself; ACS is compiled to bytecode. There’s even at least one alternative language that compiles to the same bytecode, which is interesting.

The other big feature is DECORATE, a mostly-declarative mostly-interpreted language for defining new kinds of objects. It’s a fairly direct reflection of how Doom actors are implemented, which is in terms of states. In Doom and the other commercial games, actor behavior was built into the engine, but this language has allowed almost all actors to be extracted as text files instead. For example, the imp is implemented partly as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
  States
  {
  Spawn:
    TROO AB 10 A_Look
    Loop
  See:
    TROO AABBCCDD 3 A_Chase
    Loop
  Melee:
  Missile:
    TROO EF 8 A_FaceTarget
    TROO G 6 A_TroopAttack
    Goto See
  ...
  }

TROO is the name of the imp’s sprite “family”. A, B, and so on are individual frames. The numbers are durations in tics (35 per second). All of the A_* things (which are optional) are action functions, behavioral functions (built into the engine) that run when the actor switches to that frame. An actor starts out at its Spawn state, so an imp behaves as follows:

  • Spawn. Render as TROO frame A. (By default, action functions don’t run on the very first frame they’re spawned.)
  • Wait 10 tics.
  • Change to TROO frame B. Run A_Look, which checks to see if a player is within line of sight, and if so jumps to the See state.
  • Wait 10 tics.
  • Repeat. (This time, frame A will also run A_Look, since the imp was no longer just spawned.)

All monster and item behavior is one big state table. Even the player’s own weapons work this way, which becomes very confusing — at some points a weapon can be running two states simultaneously. Oh, and there’s A_CustomMissile for monster attacks but A_FireCustomMissile for weapon attacks, and the arguments are different, and if you mix them up you’ll get extremely confusing parse errors.

It’s a little bit of a mess. It’s fairly flexible for what it is, and has come a long way — for example, even original Doom couldn’t pass arguments to action functions (since they were just function pointers), so it had separate functions like A_TroopAttack for every monster; now that same function can be written generically. People have done some very clever things with zero-delay frames (to run multiple action functions in a row) and storing state with dummy inventory items, too. Still, it’s not quite a programming language, and it’s easy to run into walls and bizarre quirks.

When DECORATE lets you down, you have one interesting recourse: to call an ACS script!

Unfortunately, ACS also has some old limitations. The only type it truly understands is int, so you can’t manipulate an actor directly or even store one in a variable. Instead, you have to work with TIDs (“thing IDs”). Every actor has a TID (zero is special-cased to mean “no TID”), and most ACS actor-related functions are expressed in terms of TIDs. For level automation, this is fine, and probably even what you want — you can dump a group of monsters in a map, give them all a TID, and then control them as a group fairly easily.

But if you want to use ACS to enhance DECORATE, you have a bit of a problem. DECORATE defines individual actor behavior. Also, many DECORATE actors are designed independently of a map and intended to be reusable anywhere. DECORATE should thus not touch TIDs at all, because they’re really the map‘s concern, and mucking with TIDs might break map behavior… but ACS can’t refer to actors any other way. A number of action functions can, but you can’t call action functions from ACS, only DECORATE. The workarounds for this are not pretty, especially for beginners, and they’re very easy to silently get wrong.

Also, ultimately, some parts of the engine are just not accessible to either ACS or DECORATE, and neither language is particularly amenable to having them exposed. Adding more native types to ACS is rather difficult without making significant changes to both the language and bytecode, and DECORATE is barely a language at all.

Some long-awaited work is finally being done on a “ZScript”, which purports to solve all of these problems by expanding DECORATE into an entire interpreted-C++-ish scripting language with access to tons of internals. I don’t know what I think of it, and it only seems to half-solve the problem, since it doesn’t replace ACS.

Trying out Lua

Lua is supposed to be easy to embed, right? That’s the one thing it’s famous for. Before ZScript actually started to materialize, I thought I’d take a little crack at embedding a Lua interpreter and exposing some API stuff to it.

It’s not very far along yet, but it can do one thing that’s always been completely impossible in both ACS and DECORATE: print out the player’s entire inventory. You can check how many of a given item the player has in either language, but neither has a way to iterate over a collection. In Lua, it’s pretty easy.

1
2
3
4
5
6
function lua_test_script(activator, ...)
    for item, amount in pairs(activator.inventory) do
        -- This is Lua's builtin print(), so it goes to stdout
        print(item.class.name, amount)
    end
end

I made a tiny test map with a switch that tries to run the ACS script named lua_test_script. I hacked the name lookup to first look for the name in Lua’s global scope; if the function exists, it’s called immediately, and ACS isn’t consulted at all. The code above is just a regular (global) function in a regular Lua file, embedded as a lump in the map. So that was a good start, and was pretty neat to see work.

Writing the bindings

I used the bare Lua API at first. While its API is definitely very simple, actually using it to define and expose a large API in practice is kind of repetitive and error-prone, and I was never confident I was doing it quite right. It’s plain C and it works entirely through stack manipulation and it relies on a lot of casting to/from void*, so virtually anything might go wrong at any time.

I was on the cusp of writing a bunch of gross macros to automate the boring parts, and then I found sol2, which is pretty great. It makes heavy use of basically every single C++11 feature, so it’s a nightmare when it breaks (and I’ve had to track down a few bugs), but it’s expressive as hell when it works:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
lua.new_usertype<AActor>("zdoom.AActor",
    "__tostring", [](AActor& actor) { return "<actor>"; },
    // Pointer to an unbound method.  Sol automatically makes this an attribute
    // rather than a method because it takes no arguments, then wraps its
    // return value to pass it back to Lua, no manual wrapper code required.
    "class", &AActor::GetClass,
    "inventory", sol::property([](AActor& actor) -> ZLuaInventory { return ZLuaInventory(actor); }),
    // Pointers to unbound attributes.  Sol turns these into writable
    // attributes on the Lua side.
    "health", &AActor::health,
    "floorclip", &AActor::Floorclip,
    "weave_index_xy", &AActor::WeaveIndexXY,
    "weave_index_z", &AActor::WeaveIndexZ);

This is the type of the activator argument from the script above. It works via template shenanigans, so most of the work is done at compile time. AActor has a lot of properties of various types; wrapping them with the bare Lua API would’ve been awful, but wrapping them with Sol is fairly straightforward.

Lifetime

activator.inventory is a wrapper around a ZLuaInventory object, which I made up. It’s just a tiny proxy struct that tries to represent the inventory of a particular actor, because the engine itself doesn’t quite have such a concept — an actor’s “inventory” is a single item (itself an actor), and each item has a pointer to the next item in the inventory. Creating an intermediate type lets me hide that detail from Lua and pretend the inventory is a real container.

The inventory is thus not a real table; pairs() works on it because it provides the __pairs metamethod. It calls an iter method returning a closure, per Lua’s iteration API, which Sol makes just work:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
struct ZLuaInventory {
    ...
    std::function<AInventory* ()> iter()
    {
        TObjPtr<AInventory> item = this->actor->Inventory;
        return [item]() mutable {
            AInventory* ret = item;
            if (ret)
                item = ret->NextInv();
            return ret;
        };
    }
}

C++’s closures are slightly goofy and it took me a few tries to land on this, but it works.

Well, sort of.

I don’t know how I got this idea in my head, but I was pretty sure that ZDoom’s TObjPtr did reference counting and would automatically handle the lifetime problems in the above code. Eventually Lua reaps the closure, then C++ reaps the closure, then the wrapped AInventorys refcount drops, and all is well.

Turns out TObjPtr doesn’t do reference counting. Rather, all the game objects participate in tracing garbage collection. The basic idea is to start from some root object and recursively traverse all the objects reachable from that root; whatever isn’t reached is garbage and can be deleted.

Unfortunately, the Lua interpreter is not reachable from ZDoom’s own object tree. If an object ends up only being held by Lua, ZDoom will think it’s garbage and delete it prematurely, leaving a dangling reference. Those are bad.

I think I can fix without too much trouble. Sol allows customizing how it injects particular types, so I can use that for the type tree that participates in this GC scheme and keep an unordered_set of all objects that are alive in Lua. The Lua interpreter itself is already wrapped in an object that participates in the GC, so when the GC descends to the wrapper, it’s easy to tell it that that set of objects is alive. I’ll probably need to figure out read/write barriers, too, but I haven’t looked too closely at how ZDoom uses those yet. I don’t know whether it’s possible for an object to be “dead” (as in no longer usable, not just 0 health) before being reaped, but if so, I’ll need to figure out something there too.

It’s a little ironic that I have to do this weird workaround when ZDoom’s tracing garbage collector is based on… Lua’s.

ZDoom does have types I want to expose that aren’t garbage collected, but those are all map structures like sectors, which are never created or destroyed at runtime. I will have to be careful with the Lua interpreter itself to make sure those can’t live beyond the current map, but I haven’t really dealt with map changes at all yet. The ACS approach is that everything is map-local, and there’s some limited storage for preserving values across maps; I could do something similar, perhaps only allowing primitive scalars.

Asynchronicity

Another critical property of ACS scripts is that they can pause themselves. They can either wait for a set number of tics with delay(), or wait for map geometry to stop being busy with something like tagwait(). So you can raise up some stairs, wait for the stairs to finish appearing, and then open the door they lead to. Or you can simulate game rules by running a script in an infinite loop that waits for a few tics between iterations. It’s pretty handy. It’s incredibly handy. It’s non-negotiable.

Luckily, Lua can emulate this using coroutines. I implemented the delay case yesterday:

1
2
3
4
5
function lua_test_script(activator, ...)
    zprint("hey it's me what's up", ...)
    coroutine.yield("delay", 70)
    zprint("i'm back again")
end

When I press the switch, I see the first message, then there’s a two-second pause (Doom is 35fps), then I see the second message.

A lot more details need to be hammered out before this is really equivalent to what ACS can do, but the basic functionality is there. And since these are full-stack coroutines, I can trivially wrap that yield gunk in a delay(70) function, so you never have to know the difference.

Determinism

ZDoom has demos and peer-to-peer multiplayer. Both features rely critically on the game state’s unfolding exactly the same way, given the same seed and sequence of inputs.

ACS goes to great lengths to preserve this. It executes deterministically. It has very, very few ways to make decisions based on anything but the current state of the game. Netplay and demos just work; modders and map authors never have to think about it.

I don’t know if I can guarantee the same about Lua. I’d think so, but I don’t know so. Will the order of keys in a table be exactly the same on every system, for example? That’s important! Even the ACS random-number generator is deterministic.

I hope this is the case. I know some games, like Starbound, implicitly assume for multiplayer purposes that scripts will execute the same way on every system. So it’s probably fine. I do wish Lua made some sort of guarantee here, though, especially since it’s such an obvious and popular candidate for game scripting.

Savegames

ZDoom allows you to quicksave at any time.

Any time.

Not while a script is running, mind you. Script execution blocks the gameplay thread, so only one thing can actually be happening at a time. But what happens if you save while a script is in the middle of a tagwait?

The coroutine needs to be persisted, somehow. More importantly, when the game is loaded, the coroutine needs to be restored to the same state: paused in the same place, with locals set to the same values. Even if those locals were wrapped pointers to C++ objects, which now have different addresses.

Vanilla Lua has no way to do this. Vanilla Lua has a pretty poor serialization story overall — nothing is built in — which is honestly kind of shocking. People use Lua for games, right? Like, a lot? How is this not an extremely common problem?

A potential solution exists in the form of Eris, a modified Lua that does all kinds of invasive things to allow absolutely anything to be serialized. Including coroutines!

So Eris makes this at least possible. I haven’t made even the slightest attempt at using it yet, but a few gotchas already stand out to me.

For one, Eris serializes everything. Even regular ol’ functions are serialized as Lua bytecode. A naïve approach would thus end up storing a copy of the entire game script in the save file.

Eris has a thing called the “permanent object table”, which allows giving names to specific Lua values. Those values are then serialized by name instead, and the names are looked up in the same table to deserialize. So I could walk the Lua namespace myself after the initial script load and stick all reachable functions in this table to avoid having them persisted. (That won’t catch if someone loads new code during play, but that sounds like a really bad idea anyway, and I’d like to prevent it if possible.) I have to do this to some extent anyway, since Eris can’t persist the wrapped C++ functions I’m exposing to Lua. Even if a script does some incredibly fancy dynamic stuff to replace global functions with closures at runtime, that’s okay; they’ll be different functions, so Eris will fall back to serializing them.

Then when the save is reloaded, Eris will replace any captured references to a global function with the copy that already exists in the map script. ZDoom doesn’t let you load saves across different mods, so the functions should be the same. I think. Hmm, maybe I should check on exactly what the load rules are. If you can load a save against a more recent copy of a map, you’ll want to get its updated scripts, but stored closures and coroutines might be old versions, and that is probably bad. I don’t know if there’s much I can do about that, though, unless Eris can somehow save the underlying code from closures/coros as named references too.

Eris also has a mechanism for storing wrapped native objects, so all I have to worry about is translating pointers, and that’s a problem Doom has already solved (somehow). Alas, that mechanism is also accessible to pure Lua code, and the docs warn that it’s possible to get into an infinite loop when loading. I’d rather not give modders the power to fuck up a save file, so I’ll have to disable that somehow.

Finally, since Eris loads bytecode, it’s possible to do nefarious things with a specially-crafted save file. But since the save file is full of a web of pointers anyway, I suspect it’s not too hard to segfault the game with a specially-crafted save file anyway. I’ll need to look into this. Or maybe I won’t, since I don’t seriously expect this to be merged in.

Runaway scripts

Speaking of which, ACS currently has detection for “runaway scripts”, i.e. those that look like they might be stuck in an infinite loop (or are just doing a ludicrous amount of work). Since scripts are blocking, the game does not actually progress while a script is running, and a very long script would appear to freeze the game.

I think ACS does this by counting instructions. I see Lua has its own mechanism for doing that, so limiting script execution “time” shouldn’t be too hard.

Defining new actors

I want to be able to use Lua with (or instead of) DECORATE, too, but I’m a little hung up on syntax.

I do have something slightly working — I was able to create a variant imp class with a bunch more health from Lua, then spawn it and fight it. Also, I did it at runtime, which is probably bad — I don’t know that there’s any way to destroy an actor class, so having them be map-scoped makes no sense.

That could actually pose a bit of a problem. The Lua interpreter should be scoped to a single map, but actor classes are game-global. Do they live in separate interpreters? That seems inconvenient. I could load the game-global stuff, take an internal-only snapshot of the interpreter with Lua (bytecode and all), and then restore it at the beginning of each level? Hm, then what happens if you capture a reference to an actor method in a save file…? Christ.

I could consider making the interpreter global and doing black magic to replace all map objects with nil when changing maps, but I don’t think that can possibly work either. ZDoom has hubs — levels that can be left and later revisited, preserving their state just like with a save — and that seems at odds with having a single global interpreter whose state persists throughout the game.

Er, anyway. So, the problem with syntax is that DECORATEs own syntax is extremely compact and designed for its very specific goal of state tables. Even ZScript appears to preserve the state table syntax, though it lets you write your own action functions or just provide a block of arbitrary code. Here’s a short chunk of the imp implementation again, for reference.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
  States
  {
  Spawn:
    TROO AB 10 A_Look
    Loop
  See:
    TROO AABBCCDD 3 A_Chase
    Loop
  ...
  }

Some tricky parts that stand out to me:

  • Labels are important, since these are state tables, and jumping to a particular state is very common. It’s tempting to use Lua coroutines here somehow, but short of using a lot of goto in Lua code (yikes!), jumping around arbitrarily doesn’t work. Also, it needs to be possible to tell an actor to jump to a particular state from outside — that’s how A_Look works, and there’s even an ACS function to do it manually.

  • Aside from being shorthand, frames are fine. Though I do note that hacks like AABBCCDD 3 are relatively common. The actual animation that’s wanted here is ABCD 6, but because animation and behavior are intertwined, the labels need to be repeated to run the action function more often. I wonder if it’s desirable to be able to separate display and behavior?

  • The durations seem straightforward, but they can actually be a restricted kind of expression as well. So just defining them as data in a table doesn’t quite work.

  • This example doesn’t have any, but states can also have a number of flags, indicated by keywords after the duration. (Slightly ambiguous, since there’s nothing strictly distinguishing them from action functions.) Bright, for example, is a common flag on projectiles, weapons, and important pickups; it causes the sprite to be drawn fullbright during that frame.

  • Obviously, actor behavior is a big part of the game sim, so ideally it should require dipping into Lua-land as little as possible.

Ideas I’ve had include the following.

Emulate state tables with arguments? A very straightforward way to do the above would be to just, well, cram it into one big table.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
define_actor{
    ...
    states = {
        'Spawn:',
        'TROO', 'AB', 10, A_Look,
        'loop',
        'See:',
        'TROO', 'AABBCCDD', 3, A_Chase,
        'loop',
        ...
    },
}

It would work, technically, I guess, except for non-literal durations, but I’d basically just be exposing the DECORATE parser from Lua and it would be pretty ridiculous.

Keep the syntax, but allow calling Lua from it? DECORATE is okay, for the most part. For simple cases, it’s great, even. Would it be good enough to be able to write new action functions in Lua? Maybe. Your behavior would be awkwardly split between Lua and DECORATE, though, which doesn’t seem ideal. But it would be the most straightforward approach, and it would completely avoid questions of how to emulate labels and state counts.

As an added benefit, this would keep DECORATE almost-purely declarative — which means editor tools could still reliably parse it and show you previews of custom objects.

Split animation from behavior? This could go several ways, but the most obvious to me is something like:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
define_actor{
    ...
    states = {
        spawn = function(self)
            self:set_animation('AB', 10)
            while true do
                A_Look(self)
                delay(10)
            end
        end,
        see = function(self)
            self:set_animation('ABCD', 6)
            while true do
                A_Chase(self)
                delay(3)
            end
        end,
    },
}

This raises plenty of other API questions, like how to wait until an animation has finished or how to still do work on a specific frame, but I think those are fairly solvable. The big problems are that it’s very much not declarative, and it ends up being rather wordier. It’s not all boilerplate, though; it’s fairly straightforward. I see some value in having state delays and level script delays work the same way, too. And in some cases, you have only an animation with no code at all, so the heavier use of Lua should balance out. I don’t know.

A more practical problem is that, currently, it’s possible to jump to an arbitrary number of states past a given label, and that would obviously make no sense with this approach. It’s pretty rare and pretty unreadable, so maybe that’s okay. Also, labels aren’t blocks, so it’s entirely possible to have labels that don’t end with a keyword like loop and instead carry straight on into the next label — but those are usually used for logic more naturally expressed as for or while, so again, maybe losing that ability is okay.

Or… perhaps it makes sense to do both of these last two approaches? Built-in classes should stay as DECORATE anyway, so that existing code can still inherit from them and perform jumps with offsets, but new code could go entirely Lua for very complex actors.

Alas, this is probably one of those questions that won’t have an obvious answer unless I just build several approaches and port some non-trivial stuff to them to see how they feel.

And further

An enduring desire among ZDoom nerds has been the ability to write custom “thinkers”. Thinkers are really anything that gets to act each tic, but the word also specifically refers to the logic responsible for moving floors, opening doors, changing light levels, and so on. Exposing those more directly to Lua, and letting you write your own, would be pretty interesting.

Anyway

I don’t know if I’ll do all of this. I somewhat doubt it, in fact. I pick it up for half a day every few weeks to see what more I can make it do, just because it’s interesting. It has virtually no chance of being upstreamed anyway (the only active maintainer hates Lua, and thinks poorly of dynamic languages in general; plus, it’s redundant with ZScript) and I don’t really want to maintain my own yet another Doom fork, so I don’t expect it to ever be a serious project.

The source code for what I’ve done so far is available, but it’s brittle and undocumented, so I’m not going to tell you where to find it. If it gets far enough along to be useful as more than a toy, I’ll make a slightly bigger deal about it.

Embedding Lua in ZDoom

Post Syndicated from Eevee original https://eev.ee/blog/2016/11/26/embedding-lua-in-zdoom/

I’ve spent a little time trying to embed a Lua interpreter in ZDoom. I didn’t get too far yet; it’s just an experimental thing I poke at every once and a while. The existing pile of constraints makes it an interesting problem, though.

Background

ZDoom is a “source port” (read: fork) of the Doom engine, with all the changes from the commercial forks merged in (mostly Heretic, Hexen, Strife), and a lot of internal twiddles exposed. It has a variety of mechanisms for customizing game behavior; two are major standouts.

One is ACS, a vaguely C-ish language inherited from Hexen. It’s mostly used to automate level behavior — at the simplest, by having a single switch perform multiple actions. It supports the usual loops and conditionals, it can store data persistently, and ZDoom exposes a number of functions to it for inspecting and altering the state of the world, so it can do some neat tricks. Here’s an arbitrary script from my DUMP2 map.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
script "open_church_door" (int tag)
{
    // Open the door more quickly on easier skill levels, so running from the
    // arch-vile is a more viable option
    int skill = GameSkill();
    int speed;
    if (skill < SKILL_NORMAL)
        speed = 64;  // blazing door speed
    else if (skill == SKILL_NORMAL)
        speed = 16;  // normal door speed
    else
        speed = 8;  // very dramatic door speed

    Door_Raise(tag, speed, 68);  // double usual delay
}

However, ZDoom doesn’t actually understand the language itself; ACS is compiled to bytecode. There’s even at least one alternative language that compiles to the same bytecode, which is interesting.

The other big feature is DECORATE, a mostly-declarative mostly-interpreted language for defining new kinds of objects. It’s a fairly direct reflection of how Doom actors are implemented, which is in terms of states. In Doom and the other commercial games, actor behavior was built into the engine, but this language has allowed almost all actors to be extracted as text files instead. For example, the imp is implemented partly as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
  States
  {
  Spawn:
    TROO AB 10 A_Look
    Loop
  See:
    TROO AABBCCDD 3 A_Chase
    Loop
  Melee:
  Missile:
    TROO EF 8 A_FaceTarget
    TROO G 6 A_TroopAttack
    Goto See
  ...
  }

TROO is the name of the imp’s sprite “family”. A, B, and so on are individual frames. The numbers are durations in tics (35 per second). All of the A_* things (which are optional) are action functions, behavioral functions (built into the engine) that run when the actor switches to that frame. An actor starts out at its Spawn state, so an imp behaves as follows:

  • Spawn. Render as TROO frame A. (By default, action functions don’t run on the very first frame they’re spawned.)
  • Wait 10 tics.
  • Change to TROO frame B. Run A_Look, which checks to see if a player is within line of sight, and if so jumps to the See state.
  • Wait 10 tics.
  • Repeat. (This time, frame A will also run A_Look, since the imp was no longer just spawned.)

All monster and item behavior is one big state table. Even the player’s own weapons work this way, which becomes very confusing — at some points a weapon can be running two states simultaneously. Oh, and there’s A_CustomMissile for monster attacks but A_FireCustomMissile for weapon attacks, and the arguments are different, and if you mix them up you’ll get extremely confusing parse errors.

It’s a little bit of a mess. It’s fairly flexible for what it is, and has come a long way — for example, even original Doom couldn’t pass arguments to action functions (since they were just function pointers), so it had separate functions like A_TroopAttack for every monster; now that same function can be written generically. People have done some very clever things with zero-delay frames (to run multiple action functions in a row) and storing state with dummy inventory items, too. Still, it’s not quite a programming language, and it’s easy to run into walls and bizarre quirks.

When DECORATE lets you down, you have one interesting recourse: to call an ACS script!

Unfortunately, ACS also has some old limitations. The only type it truly understands is int, so you can’t manipulate an actor directly or even store one in a variable. Instead, you have to work with TIDs (“thing IDs”). Every actor has a TID (zero is special-cased to mean “no TID”), and most ACS actor-related functions are expressed in terms of TIDs. For level automation, this is fine, and probably even what you want — you can dump a group of monsters in a map, give them all a TID, and then control them as a group fairly easily.

But if you want to use ACS to enhance DECORATE, you have a bit of a problem. DECORATE defines individual actor behavior. Also, many DECORATE actors are designed independently of a map and intended to be reusable anywhere. DECORATE should thus not touch TIDs at all, because they’re really the map‘s concern, and mucking with TIDs might break map behavior… but ACS can’t refer to actors any other way. A number of action functions can, but you can’t call action functions from ACS, only DECORATE. The workarounds for this are not pretty, especially for beginners, and they’re very easy to silently get wrong.

Also, ultimately, some parts of the engine are just not accessible to either ACS or DECORATE, and neither language is particularly amenable to having them exposed. Adding more native types to ACS is rather difficult without making significant changes to both the language and bytecode, and DECORATE is barely a language at all.

Some long-awaited work is finally being done on a “ZScript”, which purports to solve all of these problems by expanding DECORATE into an entire interpreted-C++-ish scripting language with access to tons of internals. I don’t know what I think of it, and it only seems to half-solve the problem, since it doesn’t replace ACS.

Trying out Lua

Lua is supposed to be easy to embed, right? That’s the one thing it’s famous for. Before ZScript actually started to materialize, I thought I’d take a little crack at embedding a Lua interpreter and exposing some API stuff to it.

It’s not very far along yet, but it can do one thing that’s always been completely impossible in both ACS and DECORATE: print out the player’s entire inventory. You can check how many of a given item the player has in either language, but neither has a way to iterate over a collection. In Lua, it’s pretty easy.

1
2
3
4
5
6
function lua_test_script(activator, ...)
    for item, amount in pairs(activator.inventory) do
        -- This is Lua's builtin print(), so it goes to stdout
        print(item.class.name, amount)
    end
end

I made a tiny test map with a switch that tries to run the ACS script named lua_test_script. I hacked the name lookup to first look for the name in Lua’s global scope; if the function exists, it’s called immediately, and ACS isn’t consulted at all. The code above is just a regular (global) function in a regular Lua file, embedded as a lump in the map. So that was a good start, and was pretty neat to see work.

Writing the bindings

I used the bare Lua API at first. While its API is definitely very simple, actually using it to define and expose a large API in practice is kind of repetitive and error-prone, and I was never confident I was doing it quite right. It’s plain C and it works entirely through stack manipulation and it relies on a lot of casting to/from void*, so virtually anything might go wrong at any time.

I was on the cusp of writing a bunch of gross macros to automate the boring parts, and then I found sol2, which is pretty great. It makes heavy use of basically every single C++11 feature, so it’s a nightmare when it breaks (and I’ve had to track down a few bugs), but it’s expressive as hell when it works:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
lua.new_usertype<AActor>("zdoom.AActor",
    "__tostring", [](AActor& actor) { return "<actor>"; },
    // Pointer to an unbound method.  Sol automatically makes this an attribute
    // rather than a method because it takes no arguments, then wraps its
    // return value to pass it back to Lua, no manual wrapper code required.
    "class", &AActor::GetClass,
    "inventory", sol::property([](AActor& actor) -> ZLuaInventory { return ZLuaInventory(actor); }),
    // Pointers to unbound attributes.  Sol turns these into writable
    // attributes on the Lua side.
    "health", &AActor::health,
    "floorclip", &AActor::Floorclip,
    "weave_index_xy", &AActor::WeaveIndexXY,
    "weave_index_z", &AActor::WeaveIndexZ);

This is the type of the activator argument from the script above. It works via template shenanigans, so most of the work is done at compile time. AActor has a lot of properties of various types; wrapping them with the bare Lua API would’ve been awful, but wrapping them with Sol is fairly straightforward.

Lifetime

activator.inventory is a wrapper around a ZLuaInventory object, which I made up. It’s just a tiny proxy struct that tries to represent the inventory of a particular actor, because the engine itself doesn’t quite have such a concept — an actor’s “inventory” is a single item (itself an actor), and each item has a pointer to the next item in the inventory. Creating an intermediate type lets me hide that detail from Lua and pretend the inventory is a real container.

The inventory is thus not a real table; pairs() works on it because it provides the __pairs metamethod. It calls an iter method returning a closure, per Lua’s iteration API, which Sol makes just work:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
struct ZLuaInventory {
    ...
    std::function<AInventory* ()> iter()
    {
        TObjPtr<AInventory> item = this->actor->Inventory;
        return [item]() mutable {
            AInventory* ret = item;
            if (ret)
                item = ret->NextInv();
            return ret;
        };
    }
}

C++’s closures are slightly goofy and it took me a few tries to land on this, but it works.

Well, sort of.

I don’t know how I got this idea in my head, but I was pretty sure that ZDoom’s TObjPtr did reference counting and would automatically handle the lifetime problems in the above code. Eventually Lua reaps the closure, then C++ reaps the closure, then the wrapped AInventorys refcount drops, and all is well.

Turns out TObjPtr doesn’t do reference counting. Rather, all the game objects participate in tracing garbage collection. The basic idea is to start from some root object and recursively traverse all the objects reachable from that root; whatever isn’t reached is garbage and can be deleted.

Unfortunately, the Lua interpreter is not reachable from ZDoom’s own object tree. If an object ends up only being held by Lua, ZDoom will think it’s garbage and delete it prematurely, leaving a dangling reference. Those are bad.

I think I can fix without too much trouble. Sol allows customizing how it injects particular types, so I can use that for the type tree that participates in this GC scheme and keep an unordered_set of all objects that are alive in Lua. The Lua interpreter itself is already wrapped in an object that participates in the GC, so when the GC descends to the wrapper, it’s easy to tell it that that set of objects is alive. I’ll probably need to figure out read/write barriers, too, but I haven’t looked too closely at how ZDoom uses those yet. I don’t know whether it’s possible for an object to be “dead” (as in no longer usable, not just 0 health) before being reaped, but if so, I’ll need to figure out something there too.

It’s a little ironic that I have to do this weird workaround when ZDoom’s tracing garbage collector is based on… Lua’s.

ZDoom does have types I want to expose that aren’t garbage collected, but those are all map structures like sectors, which are never created or destroyed at runtime. I will have to be careful with the Lua interpreter itself to make sure those can’t live beyond the current map, but I haven’t really dealt with map changes at all yet. The ACS approach is that everything is map-local, and there’s some limited storage for preserving values across maps; I could do something similar, perhaps only allowing primitive scalars.

Asynchronicity

Another critical property of ACS scripts is that they can pause themselves. They can either wait for a set number of tics with delay(), or wait for map geometry to stop being busy with something like tagwait(). So you can raise up some stairs, wait for the stairs to finish appearing, and then open the door they lead to. Or you can simulate game rules by running a script in an infinite loop that waits for a few tics between iterations. It’s pretty handy. It’s incredibly handy. It’s non-negotiable.

Luckily, Lua can emulate this using coroutines. I implemented the delay case yesterday:

1
2
3
4
5
function lua_test_script(activator, ...)
    zprint("hey it's me what's up", ...)
    coroutine.yield("delay", 70)
    zprint("i'm back again")
end

When I press the switch, I see the first message, then there’s a two-second pause (Doom is 35fps), then I see the second message.

A lot more details need to be hammered out before this is really equivalent to what ACS can do, but the basic functionality is there. And since these are full-stack coroutines, I can trivially wrap that yield gunk in a delay(70) function, so you never have to know the difference.

Determinism

ZDoom has demos and peer-to-peer multiplayer. Both features rely critically on the game state’s unfolding exactly the same way, given the same seed and sequence of inputs.

ACS goes to great lengths to preserve this. It executes deterministically. It has very, very few ways to make decisions based on anything but the current state of the game. Netplay and demos just work; modders and map authors never have to think about it.

I don’t know if I can guarantee the same about Lua. I’d think so, but I don’t know so. Will the order of keys in a table be exactly the same on every system, for example? That’s important! Even the ACS random-number generator is deterministic.

I hope this is the case. I know some games, like Starbound, implicitly assume for multiplayer purposes that scripts will execute the same way on every system. So it’s probably fine. I do wish Lua made some sort of guarantee here, though, especially since it’s such an obvious and popular candidate for game scripting.

Savegames

ZDoom allows you to quicksave at any time.

Any time.

Not while a script is running, mind you. Script execution blocks the gameplay thread, so only one thing can actually be happening at a time. But what happens if you save while a script is in the middle of a tagwait?

The coroutine needs to be persisted, somehow. More importantly, when the game is loaded, the coroutine needs to be restored to the same state: paused in the same place, with locals set to the same values. Even if those locals were wrapped pointers to C++ objects, which now have different addresses.

Vanilla Lua has no way to do this. Vanilla Lua has a pretty poor serialization story overall — nothing is built in — which is honestly kind of shocking. People use Lua for games, right? Like, a lot? How is this not an extremely common problem?

A potential solution exists in the form of Eris, a modified Lua that does all kinds of invasive things to allow absolutely anything to be serialized. Including coroutines!

So Eris makes this at least possible. I haven’t made even the slightest attempt at using it yet, but a few gotchas already stand out to me.

For one, Eris serializes everything. Even regular ol’ functions are serialized as Lua bytecode. A naïve approach would thus end up storing a copy of the entire game script in the save file.

Eris has a thing called the “permanent object table”, which allows giving names to specific Lua values. Those values are then serialized by name instead, and the names are looked up in the same table to deserialize. So I could walk the Lua namespace myself after the initial script load and stick all reachable functions in this table to avoid having them persisted. (That won’t catch if someone loads new code during play, but that sounds like a really bad idea anyway, and I’d like to prevent it if possible.) I have to do this to some extent anyway, since Eris can’t persist the wrapped C++ functions I’m exposing to Lua. Even if a script does some incredibly fancy dynamic stuff to replace global functions with closures at runtime, that’s okay; they’ll be different functions, so Eris will fall back to serializing them.

Then when the save is reloaded, Eris will replace any captured references to a global function with the copy that already exists in the map script. ZDoom doesn’t let you load saves across different mods, so the functions should be the same. I think. Hmm, maybe I should check on exactly what the load rules are. If you can load a save against a more recent copy of a map, you’ll want to get its updated scripts, but stored closures and coroutines might be old versions, and that is probably bad. I don’t know if there’s much I can do about that, though, unless Eris can somehow save the underlying code from closures/coros as named references too.

Eris also has a mechanism for storing wrapped native objects, so all I have to worry about is translating pointers, and that’s a problem Doom has already solved (somehow). Alas, that mechanism is also accessible to pure Lua code, and the docs warn that it’s possible to get into an infinite loop when loading. I’d rather not give modders the power to fuck up a save file, so I’ll have to disable that somehow.

Finally, since Eris loads bytecode, it’s possible to do nefarious things with a specially-crafted save file. But since the save file is full of a web of pointers anyway, I suspect it’s not too hard to segfault the game with a specially-crafted save file anyway. I’ll need to look into this. Or maybe I won’t, since I don’t seriously expect this to be merged in.

Runaway scripts

Speaking of which, ACS currently has detection for “runaway scripts”, i.e. those that look like they might be stuck in an infinite loop (or are just doing a ludicrous amount of work). Since scripts are blocking, the game does not actually progress while a script is running, and a very long script would appear to freeze the game.

I think ACS does this by counting instructions. I see Lua has its own mechanism for doing that, so limiting script execution “time” shouldn’t be too hard.

Defining new actors

I want to be able to use Lua with (or instead of) DECORATE, too, but I’m a little hung up on syntax.

I do have something slightly working — I was able to create a variant imp class with a bunch more health from Lua, then spawn it and fight it. Also, I did it at runtime, which is probably bad — I don’t know that there’s any way to destroy an actor class, so having them be map-scoped makes no sense.

That could actually pose a bit of a problem. The Lua interpreter should be scoped to a single map, but actor classes are game-global. Do they live in separate interpreters? That seems inconvenient. I could load the game-global stuff, take an internal-only snapshot of the interpreter with Lua (bytecode and all), and then restore it at the beginning of each level? Hm, then what happens if you capture a reference to an actor method in a save file…? Christ.

I could consider making the interpreter global and doing black magic to replace all map objects with nil when changing maps, but I don’t think that can possibly work either. ZDoom has hubs — levels that can be left and later revisited, preserving their state just like with a save — and that seems at odds with having a single global interpreter whose state persists throughout the game.

Er, anyway. So, the problem with syntax is that DECORATEs own syntax is extremely compact and designed for its very specific goal of state tables. Even ZScript appears to preserve the state table syntax, though it lets you write your own action functions or just provide a block of arbitrary code. Here’s a short chunk of the imp implementation again, for reference.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
  States
  {
  Spawn:
    TROO AB 10 A_Look
    Loop
  See:
    TROO AABBCCDD 3 A_Chase
    Loop
  ...
  }

Some tricky parts that stand out to me:

  • Labels are important, since these are state tables, and jumping to a particular state is very common. It’s tempting to use Lua coroutines here somehow, but short of using a lot of goto in Lua code (yikes!), jumping around arbitrarily doesn’t work. Also, it needs to be possible to tell an actor to jump to a particular state from outside — that’s how A_Look works, and there’s even an ACS function to do it manually.

  • Aside from being shorthand, frames are fine. Though I do note that hacks like AABBCCDD 3 are relatively common. The actual animation that’s wanted here is ABCD 6, but because animation and behavior are intertwined, the labels need to be repeated to run the action function more often. I wonder if it’s desirable to be able to separate display and behavior?

  • The durations seem straightforward, but they can actually be a restricted kind of expression as well. So just defining them as data in a table doesn’t quite work.

  • This example doesn’t have any, but states can also have a number of flags, indicated by keywords after the duration. (Slightly ambiguous, since there’s nothing strictly distinguishing them from action functions.) Bright, for example, is a common flag on projectiles, weapons, and important pickups; it causes the sprite to be drawn fullbright during that frame.

  • Obviously, actor behavior is a big part of the game sim, so ideally it should require dipping into Lua-land as little as possible.

Ideas I’ve had include the following.

Emulate state tables with arguments? A very straightforward way to do the above would be to just, well, cram it into one big table.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
define_actor{
    ...
    states = {
        'Spawn:',
        'TROO', 'AB', 10, A_Look,
        'loop',
        'See:',
        'TROO', 'AABBCCDD', 3, A_Chase,
        'loop',
        ...
    },
}

It would work, technically, I guess, except for non-literal durations, but I’d basically just be exposing the DECORATE parser from Lua and it would be pretty ridiculous.

Keep the syntax, but allow calling Lua from it? DECORATE is okay, for the most part. For simple cases, it’s great, even. Would it be good enough to be able to write new action functions in Lua? Maybe. Your behavior would be awkwardly split between Lua and DECORATE, though, which doesn’t seem ideal. But it would be the most straightforward approach, and it would completely avoid questions of how to emulate labels and state counts.

As an added benefit, this would keep DECORATE almost-purely declarative — which means editor tools could still reliably parse it and show you previews of custom objects.

Split animation from behavior? This could go several ways, but the most obvious to me is something like:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
define_actor{
    ...
    states = {
        spawn = function(self)
            self:set_animation('AB', 10)
            while true do
                A_Look(self)
                delay(10)
            end
        end,
        see = function(self)
            self:set_animation('ABCD', 6)
            while true do
                A_Chase(self)
                delay(3)
            end
        end,
    },
}

This raises plenty of other API questions, like how to wait until an animation has finished or how to still do work on a specific frame, but I think those are fairly solvable. The big problems are that it’s very much not declarative, and it ends up being rather wordier. It’s not all boilerplate, though; it’s fairly straightforward. I see some value in having state delays and level script delays work the same way, too. And in some cases, you have only an animation with no code at all, so the heavier use of Lua should balance out. I don’t know.

A more practical problem is that, currently, it’s possible to jump to an arbitrary number of states past a given label, and that would obviously make no sense with this approach. It’s pretty rare and pretty unreadable, so maybe that’s okay. Also, labels aren’t blocks, so it’s entirely possible to have labels that don’t end with a keyword like loop and instead carry straight on into the next label — but those are usually used for logic more naturally expressed as for or while, so again, maybe losing that ability is okay.

Or… perhaps it makes sense to do both of these last two approaches? Built-in classes should stay as DECORATE anyway, so that existing code can still inherit from them and perform jumps with offsets, but new code could go entirely Lua for very complex actors.

Alas, this is probably one of those questions that won’t have an obvious answer unless I just build several approaches and port some non-trivial stuff to them to see how they feel.

And further

An enduring desire among ZDoom nerds has been the ability to write custom “thinkers”. Thinkers are really anything that gets to act each tic, but the word also specifically refers to the logic responsible for moving floors, opening doors, changing light levels, and so on. Exposing those more directly to Lua, and letting you write your own, would be pretty interesting.

Anyway

I don’t know if I’ll do all of this. I somewhat doubt it, in fact. I pick it up for half a day every few weeks to see what more I can make it do, just because it’s interesting. It has virtually no chance of being upstreamed anyway (the only active maintainer hates Lua, and thinks poorly of dynamic languages in general; plus, it’s redundant with ZScript) and I don’t really want to maintain my own yet another Doom fork, so I don’t expect it to ever be a serious project.

The source code for what I’ve done so far is available, but it’s brittle and undocumented, so I’m not going to tell you where to find it. If it gets far enough along to be useful as more than a toy, I’ll make a slightly bigger deal about it.

Weekly roundup: Screw it

Post Syndicated from Eevee original https://eev.ee/dev/2016/11/21/weekly-roundup-screw-it/

November is a disaster and I’ve given up on it.

  • runed awakening: I found a decent DAG diagrammer and plotted out most of the puzzles and finally had visual confirmation that a few parts of the game are lacking. Haven’t yet figured out how to address that.

  • blog: I finished an article for SitePoint. Also I wrote a thing about iteration.

  • veekun: Playing Pokémon has made me interested in veekun work again, so I did some good ergonomic work on the YAML thing, which is starting to look like it might end up kind of sensible.

  • zdoom: I had a peek at my zdoom-lua branch again, dug into object lifetime, and discovered a world of pain.

Everything is still distracting, and Pokémon came out, so.

Weekly roundup: Screw it

Post Syndicated from Eevee original https://eev.ee/dev/2016/11/21/weekly-roundup-screw-it/

November is a disaster and I’ve given up on it.

  • runed awakening: I found a decent DAG diagrammer and plotted out most of the puzzles and finally had visual confirmation that a few parts of the game are lacking. Haven’t yet figured out how to address that.

  • blog: I finished an article for SitePoint. Also I wrote a thing about iteration.

  • veekun: Playing Pokémon has made me interested in veekun work again, so I did some good ergonomic work on the YAML thing, which is starting to look like it might end up kind of sensible.

  • zdoom: I had a peek at my zdoom-lua branch again, dug into object lifetime, and discovered a world of pain.

Everything is still distracting, and Pokémon came out, so.

The Raspberry Pi-powered loom

Post Syndicated from Liz Upton original https://www.raspberrypi.org/blog/the-raspberry-pi-powered-loom/

We’re a small organisation full of makers, and I think at least two of us own a hand loom for weaving textiles. (One of the reasons I enjoy the TV show Vikings so much is the casual looming that’s going on as backdrop in many of the indoor scenes – the textile sort, not the impending-doom sort, although there’s plenty of that too.)

siggy laergatha loom

Siggy and Laergatha (personal role model) get down to a spot of light weaving before commencing to crush skulls and pillage.

Here in the 21st century, Lorna and I use hand looms because powered looms are very expensive. They’re also usually pretty enormous, being meant for enterprise rather than home use. This is pesky, because there’s a lot of repetitive action involved, which can be hell on the carpal tunnels; weaving can be slow, tough work.

loom

Suspicious automation

Enter the Raspberry Pi.

Fred Hoefler has taken a desktop loom and added a Raspberry Pi to automate it. (Your computer’s fine: this video has no sound.)

Loom Operation

The general sequence of events for running my Raspberry Pi controlled loom. The project was really a proof of concept idea rather than an actual production model. This video is intended to supplement my blog at www.photographic-perspectives.com Sorry, there is not audio with this.

Fred wrote about the project on his website, explaining that he came up with the idea for very personal reasons. His wife Gina has been a weaver for 30 years, but she began to experience difficulties with the physical aspects of using her loom as she grew older. Conversations with other unwillingly retired weavers told Fred that Gina’s situation was not uncommon, and led him to design something to help. His device is intended to help older weavers who have trouble with the hard work of throwing the shuttle and holding down the pedals. Assistive looms cost upwards of $10,000: Fred’s solution comes in at a tidy $150, factoring in loom, Pi, and some motors from Amazon. So this isn’t for hobbyists like me: this loom can be a way for people whose livelihoods depend on being able to weave to continue working long after they might have had to retire.

One of the most satisfying things about the Raspberry Pi for me is its power to drive cost out of devices like this, and to change the way we work. This is a simple build, but it has so much potential to keep someone’s income flowing: we hope to see more as Fred develops the project.

The post The Raspberry Pi-powered loom appeared first on Raspberry Pi.

Weekly roundup: National Novelty Writing Month

Post Syndicated from Eevee original https://eev.ee/dev/2016/11/07/weekly-roundup-national-novelty-writing-month/

Inktober is a distant memory.

Now it’s time for NaNoWriMo! Almost. I don’t have any immediate interest in writing a novel, but I do have plenty of other stuff that needs writing — blog posts, my book, Runed Awakening, etc. So I’m going to try to write 100,000 words this month, spread across whatever.

Rules:

  1. I’m only measuring, like, works. I’ll count this page, as short as it is, because it’s still a single self-contained thing that took some writing effort. But no tweets or IRC or the like.

  2. I’m counting with vim’s g C-g or wc -w, whichever is more convenient. The former is easier for single files I edit in vim; the latter is easier for multiple files or stuff I edit outside of vim.

  3. I’m making absolutely zero effort to distinguish between English text, code, comments, etc.; whatever the word count is, that’s what it is. So code snippets in the book will count, as will markup in blog posts. Runed Awakening is a weird case, but I’m choosing to count it because it’s inherently a text-based game, plus it’s written in a prosaic language. On the other hand, dialogue for Isaac HD does not count, because it’s a few bits of text in what is otherwise just a Lua codebase.

  4. Only daily net change counts. This rule punishes me for editing, but that’s the entire point of NaNoWriMo’s focus on word count: to get something written rather than linger on a section forever and edit it to death. I tend to do far too much of the latter.

    This rule already bit me on day one, where I made some significant progress on Runed Awakening but ended up with a net word count of -762 because it involved some serious refactoring. Oops. Turns out word-counting code is an even worse measure of productivity than line-counting code.

These rules are specifically crafted to nudge me into working a lot more on my book and Runed Awakening, those two things I’d hoped to get a lot further on in the last three months. And unlike Inktober, blog posts contribute towards my preposterous goal rather than being at odds with it.

With one week down, so far I’m at +8077 words. I got off to a pretty slow (negative, in fact) start, and then spent a day out of action from an ear infection, so I’m a bit behind. Hoping I can still catch up as I get used to this whole “don’t rewrite the same paragraph over and over for hours” approach.

  • art: Last couple ink drawings of Pokémon, hallelujah. I made a montage of them all, too.

    I drew Momo (the cat from Google’s Halloween doodle game) alongside Isaac and it came out spectacularly well.

    I finally posted the loophole commission.

    I posted a little “what type am I” meme on Twitter and drew some of the interesting responses. I intended to draw a couple more, but then I got knocked on my ass and my brain stopped working. I still might get back to them later.

  • blog: I posted an extremely thorough teardown of JavaScript. That might be cheating, but it’s okay, because I love cheating.

    Wrote a whole lot about Java.

  • doom: I did another speedmap. I haven’t released the last two yet; I want to do a couple more and release them as a set.

  • blog: I wrote about game accessibility, which touched on those speedmaps.

  • runed awakening: I realized I didn’t need all the complexity of (and fallout caused by) the dialogue extension I was using, so I ditched it in favor of something much simpler. I cleaned up some stuff, fixed some stuff, improved some stuff, and started on some stuff. You know.

  • book: I’m working on the PICO-8 chapter, since I’ve actually finished the games it describes. I’m having to speedily reconstruct the story of how I wrote Under Construction, which is interesting. I hope it still comes out like a story and not a tutorial.

As for the three big things, well, they sort of went down the drain. I thought they might; I don’t tend to be very good at sticking with the same thing for a long and contiguous block of time. I’m still making steady progress on all of them, though, and I did some other interesting stuff in the last three months, so I’m satisfied regardless.

With November devoted almost exclusively to writing, I’m really hoping I can finally have a draft chapter of the book ready for Patreon by the end of the month. That $4 tier has kinda been languishing, sorry.