Lua

[KeksDose]

It's the lightning, too - can it be made to do the blocky lighting like the software renderer?

There's also a 'Depth Fog' Option in the 'GL Options'. And if the fog is way too far away, you can use 'GL_DistFog' in the console to make it a bit more visible. There you go. It isn't the Doom Fog, though, but everything brights up the closer you get.

[randi]

He hasn't made a single comment so far. Well, there's always GZDoom, right?

I haven't looked at it to be honest. I just want to get the website stuff out of the way without being distracted by Doom code. I've only been keeping a cursory track of what you've committed to SVN. I honestly hope I won't have too much trouble merging my current work-in-progress changes with all the commits you've done in the last four months.

That said, I still want the experience of writing my own scripting language. I tried eight years ago but didn't get too far, and I'd like to succeed this time around. I worry about the split that would occur if you added one thing to GZDoom and I added something else to ZDoom.

So here are the thoughts I had about what I want to see in the language before I got the notice from Lunarpages that I need to streamline my site or risk being kicked off. (Fortunately, they seem pretty lenient as long as I'm on this non-production server, but I do want to hurry up and get off, so I've not given the language much thought since then.):

• Object is a basic type for the language, as it is for Java and C#. New classes implicitly have it as an ancestor, and you are not limited to only subclassing actors.
• The language supports dynamic allocation of objects.
• An incremental garbage collector is used to reclaim dead objects. Not only does this free users from the burden of having to worry about freeing objects, but it also simplifies various implementation details. The pointer cleanup ZDoom already does is essentially a special-case mark-and-sweep single-pass garbage collector that can run as many as 35 times per second depending on the level activity. It's already known that this can cause slowdown in extreme maps. An incremental garbage collector should theoretically be able to perform better, since it can spread the collection out across multiple tics instead of doing it all at once. At worst, the performance should be about the same as what's in place now.
• Objects can also have arbitrary data associated with them by accessing them like an array:

  Code: Select all

  someobj['mydata'] = foo;

  This provides an alternative path for extending objects without subclassing, in cases where subclassing is impractical. This is mostly for the benefit of using the language as a replacement for ACS. Right now, extra data about player state that modders want to maintain is commonly kept in arrays. Wouldn't it be better if that data could be kept directly with the player? It's cleaner and, more importantly, avoids the problem of assuming a fixed maximum number of players.
  
  Perhaps there can also be syntactic sugar for this so that the above example can also be written as:

  Code: Select all

  someobj:mydata = foo;

  I'll have to see if that causes any conflicts with the grammar for labels first.
• The parser is designed to be two-pass, so it collects type information in the first pass and compiles functions in the second pass. Then you don't have to worry about the order you define classes in.
• I'm not sure it's worth the trouble of trying to retrofit this into the existing DECORATE framework. I'd much rather start off with a new file format with a clean slate, with DECORATE support maintained as-is and give modders the option to use whichever format they prefer: DECORATE for everything possible now and script for everything else. New actors defined in script could even expose methods to be used for DECORATE parsing, so you could define an actor in script and then subclass that in DECORATE.
• The VM is register-based, so there are fewer hard-to-predict indirect branches needed for executing script code. For the purposes of serialization, the registers are tagged with types, although the instructions that act on the registers don't need to actually check those types, so long as the compiler produces correct code. I had started work on the VM before the Lunarpages business came up, so some work has been done.
• A debugger interface can be added later once everything else is working.

[DoomRater]

Oooh, building new working codepointers and using them in DECORATE is always a welcome method. I look forward to Randy's ideas once the site's taken care of.

[Jim]

Object is a basic type for the language, as it is for Java and C#. New classes implicitly have it as an ancestor, and you are not limited to only subclassing actors.

C++ is more of the exception than the rule as far as NOT having a single common superclass.

[*]Objects can also have arbitrary data associated with them by accessing them like an array:

Code: Select all

someobj['mydata'] = foo;

This provides an alternative path for extending objects without subclassing, in cases where subclassing is impractical. This is mostly for the benefit of using the language as a replacement for ACS. Right now, extra data about player state that modders want to maintain is commonly kept in arrays. Wouldn't it be better if that data could be kept directly with the player? It's cleaner and, more importantly, avoids the problem of assuming a fixed maximum number of players.

I like it. It sounds like my ECMAScript suggestion (the root standard in JavaScript, Flash). This suggests an implementation similar to JavaScript, where a class is really just a hash table.

You'll want to stick with a balanced binary tree until there get to be quite a few class members. You can find the exact point where it makes sense to take on the additional overhead of hashtables by a little profiling, while also keeping in mind space and cache locality.

How do you intend to handle when someone tries to get the value of someobj['mydata'] when it is uninitialized? You can avoid a lot of messiness of ECMAScript by simply returning null and keeping 'undefined' out of the picture completely.

Perhaps there can also be syntactic sugar for this so that the above example can also be written as:

Code: Select all

someobj:mydata = foo;

I'll have to see if that causes any conflicts with the grammar for labels first.

I think it would be best to try to stick with the model of ECMAScript, where you can access either using indexing or as member objects.

[*]I'm not sure it's worth the trouble of trying to retrofit this into the existing DECORATE framework. I'd much rather start off with a new file format with a clean slate, with DECORATE support maintained as-is and give modders the option to use whichever format they prefer: DECORATE for everything possible now and script for everything else. New actors defined in script could even expose methods to be used for DECORATE parsing, so you could define an actor in script and then subclass that in DECORATE.

I agree. Layering on top of DECORATE is not the way to produce a clean language, not even if you did simply choose to accept that bad syntax that worked in the past will not anymore.

[bagheadspidey]

I'm not sure it's worth the trouble of trying to retrofit this into the existing DECORATE framework. I'd much rather start off with a new file format with a clean slate, with DECORATE support maintained as-is and give modders the option to use whichever format they prefer: DECORATE for everything possible now and script for everything else. New actors defined in script could even expose methods to be used for DECORATE parsing, so you could define an actor in script and then subclass that in DECORATE.

You may be interested in the way the Vavoom source port handles this; DECORATE is preserved alongside a fully-functional scripting language. Have a look:

---

Code: Select all

//**************************************************************************
//**
//**    ##   ##    ##    ##   ##   ####     ####   ###     ###
//**    ##   ##  ##  ##  ##   ##  ##  ##   ##  ##  ####   ####
//**     ## ##  ##    ##  ## ##  ##    ## ##    ## ## ## ## ##
//**     ## ##  ########  ## ##  ##    ## ##    ## ##  ###  ##
//**      ###   ##    ##   ###    ##  ##   ##  ##  ##       ##
//**       #    ##    ##    #      ####     ####   ##       ##
//**
//**    $Id: ZombieMan.vc 2669 2007-08-23 23:01:35Z dj_jl $
//**
//**    Copyright (C) 1999-2006 Jānis Legzdiņš
//**
//**    This program is free software; you can redistribute it and/or
//**  modify it under the terms of the GNU General Public License
//**  as published by the Free Software Foundation; either version 2
//**  of the License, or (at your option) any later version.
//**
//**    This program is distributed in the hope that it will be useful,
//**  but WITHOUT ANY WARRANTY; without even the implied warranty of
//**  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
//**  GNU General Public License for more details.
//**
//**************************************************************************

class ZombieMan : Actor
	__mobjinfo__(3004)
	__scriptid__(4);

//==========================================================================
//
//  A_PosAttack
//
//  Zombieman attack.
//
//==========================================================================

final void A_PosAttack()
{
	int damage;
	TVec dir;

	if (!Target)
		return;

	A_FaceTarget();

	AimLineAttack(dir, Angles, MISSILERANGE);
	VectorRotateAroundZ(&dir, (Random() - Random()) * 45.0 / 2.0);

	PlaySound('grunt/attack', CHAN_WEAPON);

	damage = ((P_Random() % 5) + 1) * 3;

	LineAttack(dir, MISSILERANGE, damage, BulletPuff);

	bMuzzleFlash = false;
}

//==========================================================================
//
//	NoBlockingSet
//
//==========================================================================

void NoBlockingSet()
{
	DropItem(Clip, 0, 1.0);
}

states
{
Spawn:
	POSS AB 10 A_Look
	Loop
See:
	POSS AABBCCDD 4 A_Chase
	Loop
Missile:
	POSS E 10 A_FaceTarget
	POSS F 8 A_PosAttack
	POSS E 8
	Goto See
Pain:
	POSS G 3
	POSS G 3 A_Pain
	Goto See
Death:
	POSS H 5
	POSS I 5 A_Scream
	POSS J 5 A_Fall
	POSS K 5
	POSS L -1
	Stop
XDeath:
	POSS M 5
	POSS N 5 A_XScream
	POSS O 5 A_Fall
	POSS PQRST 5
	POSS U -1
	Stop
Raise:
	POSS KJIH 5
	Goto See
}

defaultproperties
{
	Health = 20;
	GibsHealth = -20;
	Radius = 20.0;
	Height = 56.0;
	Mass = 100.0;
	StepSpeed = 8.0;
	PainChance = 0.78125;
	bSolid = true;
	bShootable = true;
	bCountKill = true;
	bMonster = true;
	bActivateMCross = true;
	bActivatePushWall = true;
	bFloorClip = true;
	DLightColour = RGB(195, 79, 0);
	SightSound = 'grunt/sight';
	ActiveSound = 'grunt/active';
	AttackSound = 'grunt/attack';
	PainSound = 'grunt/pain';
	DeathSound = 'grunt/death1';
}

edit: in case you are wondering, Actor::A_Fall calls NoBlockingSet.

[Nash]

I never realized Vavoom's actor syntax is similar to DECORATE!

[bagheadspidey]

I never realized Vavoom's actor syntax is similar to DECORATE!

Right - "Vavoom actor states" are not exactly DECORATE but it's so damn close it might as well be.

[Graf Zahl]

So here are the thoughts I had about what I want to see in the language before I got the notice from Lunarpages that I need to streamline my site or risk being kicked off. (Fortunately, they seem pretty lenient as long as I'm on this non-production server, but I do want to hurry up and get off, so I've not given the language much thought since then.):
[*]Object is a basic type for the language, as it is for Java and C#. New classes implicitly have it as an ancestor, and you are not limited to only subclassing actors.

Yes, that's how I implemented it. So far I haven't done any non-actor stuff though but it should work once I progress further.

[*]The language supports dynamic allocation of objects.

I haven't implemented that yet but it's needed for the non-actor stuff later so it's somewhere down on my to-do list.

[*]An incremental garbage collector is used to reclaim dead objects. Not only does this free users from the burden of having to worry about freeing objects, but it also simplifies various implementation details. The pointer cleanup ZDoom already does is essentially a special-case mark-and-sweep single-pass garbage collector that can run as many as 35 times per second depending on the level activity. It's already known that this can cause slowdown in extreme maps. An incremental garbage collector should theoretically be able to perform better, since it can spread the collection out across multiple tics instead of doing it all at once. At worst, the performance should be about the same as what's in place now.

A question: How do you want to handle this? Like Java where all pointer assignments are tracked and an object is freed once all are gone or more like ZDoom where you tell an object to destroy itself and then the garbage collector takes over and gets rid of the object and all references to it? I've worked enough with Java that I really don't like its method. Especially in a situation with thousands of objects it's nearly impossible to keep track of everything and leaks are inevitable. ZDoom's method with a more performance-efficient cleanup code would also mean that less changes are needed to the code.
Another issue is that right now I am very careful not to have a non-compilable version for more than a few hours so that I can test everything as quickly as possible and I have to stick closely to the way things are done in the C++ code. That also has the advantage for me that I can basically leave the code as it is and only make the necessary syntax adjustments.

[*]Objects can also have arbitrary data associated with them by accessing them like an array:

Code: Select all

someobj['mydata'] = foo;

This provides an alternative path for extending objects without subclassing, in cases where subclassing is impractical. This is mostly for the benefit of using the language as a replacement for ACS. Right now, extra data about player state that modders want to maintain is commonly kept in arrays. Wouldn't it be better if that data could be kept directly with the player? It's cleaner and, more importantly, avoids the problem of assuming a fixed maximum number of players.

That sounds like a cool idea but I haven't spent much thought about it yet because it's an extension to the existing functionality, not a replacement. But I see no reason why this can't be added.

Perhaps there can also be syntactic sugar for this so that the above example can also be written as:

Code: Select all

someobj:mydata = foo;

I'll have to see if that causes any conflicts with the grammar for labels first.

Grammar... That's the one thing where I folded early on because I never managed to get a grammar working that can handle the old DECORATE stuff without problems. That's because all the existing features cannot be parsed with a real tokenizer and I have to switch parsing modes at several points during the process in [order to keep compatibility. I am using all 3 modes of sc_man: classic, C and tokenized. Yes, I know, it's a nightmare - but necessary to handle existing WADs.

[*]The parser is designed to be two-pass, so it collects type information in the first pass and compiles functions in the second pass. Then you don't have to worry about the order you define classes in.

That's what I am doing, too. Although I made some restrictions regarding variable types so that the thing can still work within the existing DECORATE without causing conflicts. That means mostly that class and struct types have to be defined as 'class<name>' and 'struct<name>' to keep their namespace separate from DECORATE keywords. I have been thinking about relaxing this within function definitions but I havent done so yet.

[*]I'm not sure it's worth the trouble of trying to retrofit this into the existing DECORATE framework. I'd much rather start off with a new file format with a clean slate, with DECORATE support maintained as-is and give modders the option to use whichever format they prefer: DECORATE for everything possible now and script for everything else. New actors defined in script could even expose methods to be used for DECORATE parsing, so you could define an actor in script and then subclass that in DECORATE.

Mine is working within DECORATE and the problems so far are only minor. I have found only one WAD where I couldn't handle it at all due to its bad definitions which used integers where strings were needed. The old parser let these through without saying a word. Everything else prints out some warnings about unquoted strings or undefined actor classes at most and both cases can be handled without going into too much trouble.
Dehacked is working fine as well.

[*]The VM is register-based, so there are fewer hard-to-predict indirect branches needed for executing script code. For the purposes of serialization, the registers are tagged with types, although the instructions that act on the registers don't need to actually check those types, so long as the compiler produces correct code. I had started work on the VM before the Lunarpages business came up, so some work has been done.

I have used a simple stack machine but so far I haven't done anything with code that has real performance impact so I can't say much about actual speed yet.

[*]A debugger interface can be added later once everything else is working.

Yes - but I fear that this will be more work than everything else combined...


That all said, I know fully well that much of my code is quite messy and will defintely need a complete cleanup including heavy rewrites once all the needed features are working.

A few things of interest:

• How strongly would you integrate the compiler into the engine. I resolve any class and sound names at compile time if possible because these operations are needed quite frequently and not having to look up this data should certainly save some time.
• Do you consider an external compiler an option? Because I think that would make things a lot more complicated because far less things can be done during compiling. My compile time tests show that it isn't really necessary. Ok, I have an extremely fast computer (Core Quad 2.4 GHz) but for me the compile time of all the data from the g_doom, g_heretic, g_hexen, g_raven and g_strife directories plus 10% of g_shared is only 0.1 seconds when launched after a reboot of the computer and the HD cache is empty. This should mean even on slower computers that it can be done under a second.

You may be interested in the way the Vavoom source port handles this; DECORATE is preserved alongside a fully-functional scripting language. Have a look:

I noticed. He changed it for the latest version and his states syntax is a carbon copy of DECORATE. When converting all the Hexen stuff I could copy&paste most of his states definitions into my DECORATE files and it worked out of the box in most cases.

[Graf Zahl]

How do you intend to handle when someone tries to get the value of someobj['mydata'] when it is uninitialized? You can avoid a lot of messiness of ECMAScript by simply returning null and keeping 'undefined' out of the picture completely.

I think 'undefined' needs to be properly defined.
Sometimes you can't just assume a certain value as the default. The existing DECORATE treats 0 as such and the issues were so much that when I started writing my compiler my first decision was to allow default parameter values like in C++ to avoid the problem.
I don't see any difference between function parameters and custom properties. If one isn't defined the code should be able to know (in the first case by not allowing to compile and in the second by reporting it properly.)

[randi]

I like it. It sounds like my ECMAScript suggestion (the root standard in JavaScript, Flash). This suggests an implementation similar to JavaScript, where a class is really just a hash table.

I was thinking of Lua, actually. It would be a hybrid of C-like structures and Lua-like tables, so it can integrate better with the existing code.

You'll want to stick with a balanced binary tree until there get to be quite a few class members. You can find the exact point where it makes sense to take on the additional overhead of hashtables by a little profiling, while also keeping in mind space and cache locality.

Have you had a look at Lua's table algorithm? It tries to keep a hash table in a compact area of memory while also minimizing chain lengths. Seems to work pretty well to me, given Lua's ability to outperform pretty much every similar language.

How do you intend to handle when someone tries to get the value of someobj['mydata'] when it is uninitialized?

Again, like Lua: Just return nil. Setting it to nil would also remove it from the table.

A question: How do you want to handle this [garbage collection]?

Ultimately, the incremental garbage collector is responsible for reclaiming all memory. However, objects can be told to destroy themselves while references still remain. Doing so sets a flag on the object indicating it wants to be garbage. Code that uses object references needs to check that the object doesn't want to go garbage across function calls. If it does, then any references the function has need to be NULLed. (A template class ought to make all this transparent for C++, and script code would never need to know, since the VM would take care of it.) When the collector runs, either all references to the object have been NULLed, so it never encounters the object, or it sees the flag and NULLs the reference that brought it to the object without marking it as in-use. Either way, the object will be eliminated at the end of the next full collection cycle.

The VM is register-based

Another factor for this decision that I forgot to mention is that much of the compiler literature seems to be written with the assumption that a three-address machine is used for intermediate code, so it's easier to take algorithms from those books and apply them if that's what the VM is using.

How strongly would you integrate the compiler into the engine. ... Do you consider an external compiler an option?

I don't see any point in keeping them separate anymore.

Right - "Vavoom actor states" are not exactly DECORATE but it's so damn close it might as well be.

There's a big difference between a state definition and DECORATE.

[bagheadspidey]

Right - "Vavoom actor states" are not exactly DECORATE but it's so damn close it might as well be.

There's a big difference between a state definition and DECORATE.

I admit there are probably many differences I am unaware of, but the code sure looks and feels similar.

As for the js vs lua thing, my 0.02 USD (fwiw, etc): I think you should implement the underlying engine however you like, but I think that js has a friendlier syntax than lua. For example, the use of the : symbol to make static calls, the implied object-as-first-argument when making dynamic calls, functions returning multiple values, etc could be confusing to less experienced programmers. Also some operators are "weird" compared to other languages (~=). I have mixed feelings about "end" vs "}" type syntax: I personally favor the curly braces, but "end" might be friendlier for beginners I think it would actually be really cool to see a language that allows both.

I'm not sure if it your intent to make the syntax lua-like or not, but imho js has a much friendlier syntax to learn.

[Nash]

So this not going to be C-like?

if (!c_like)
{
ohnoes++;
disappointment++;
}

[bagheadspidey]

So this not going to be C-like?

if (!c_like)
{
ohnoes++;
disappointment++;
}

interestingly enough, that bit of code would be perfectly valid in js as well. I just hope we can all agree that dots are much nicer than arrows?

[Nash]

Well, in QuakeC (the only form of "advanced" programming I have experience with outside of ACS - very sad, I know) you access variables under an entity with dots. So to get the player health you'd do self.health.

How does C++ do it?

[TheDarkArchon]

self->health or (*self).health.

At least that's how ZDoom does it internally (represented in the former manner), with *self being a pointer to the actor that called the action in question.